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Chapter 1 


Introduction 


1.1 Justus 


Fourteen years ago, in the crew room of Royal Netherlands Air Force (RN- 
LAF) 301 Squadron, fellow aviator “Spank” shared the story of his newborn 
son. 


Justus arrived into this world prematurely, with a serious medical condition 
called gastroschizis. A vulnerable little patient in need of specialised care. 


When Justus was transferred to a high-end, well-equipped Neonatal Inten- 
sive Care Unit, the parents encountered clinical staff with lots of individual 
knowledge and motivation. 


But Justus was not doing well. And when complications occurred his 
parents grew increasingly anxious. 


Spank, being aviator and flight safety officer, could not help noticing threats 
to safe care: suboptimal reaction to alarms, ineffective shift handovers, 
fragile interdisciplinary teamwork, and communication techniques 
nowhere near the Air Force standards he knew so well. 


After sharing his observations with the doctor in charge, Spank provided 
information on the RNLAF's Human Factors awareness program entitled 
Crew Resource Management (CRM). This resulted in a request for an intro- 
ductory CRM-course for the clinical team involved. Our very first effort to 
adapt aviation Human Factors principles to the hospital environment was 
well received by this medical team, who embrace the CRM principles to 
date. 


Justus survived and has grown into a fine young man, doing very well at 
school. His proud parents hope that more of my fellow caregivers will learn 
from his story. 


1.2 The scope of the problem 


Medical care frequently results in unintentional harm to patients. Despite 
the bestintentions of caregivers, technical advancements, quality and safety 
programs and ever more stringent procedural guidelines, statistics do not 
seem to improve. If medical error were a disease, it would rank as the third 
leading cause of death in the US." 


Human Factors 

To err is human.’ As a result everything that a human being devises, uses, 
or does is prone to error and failure. Human Factors (HF), also known as 
ergonomics, uses scientific methods to understand better human behavior 
in relation to organizational and task factors. The aim is to improve system 
performance and prevent accidental harm. 

In clinical medicine, HF-related errors can have a major impact on patient 
safety.?* Especially in departments where high risk, time critical procedures 
on vulnerable patients are performed in a multidisciplinary team setting, 
HF are likely to play an important role. This is for instance illustrated by the 
fact that the occurrence of complications is related to outcome in critically 
ill patients.” 


Safety Climate 

As HF account for the majority of adverse events, assessing the “safety 
culture” within a healthcare organization is considered important. While 
several definitions of the term “safety culture” exist, it may best be descri- 
bed as the shared values, beliefs and principles towards safety within an 
organization.® 

Measuring safety culture is challenging and would require careful, long- 
term observations. Within the healthcare domain it may therefore be more 
appropriate to use the term “safety climate” (the daily practices and behavi- 
ors of individual healthcare professionals as well as interactions within and 
between healthcare teams), which can be determined by questionnaires. 


Evidence from other critical industries suggests that safety climate corre- 
lates with unsafe and safety-specific behavior, injury rates and accidents”? 


Crew Resource Management 
In multiple safety-critical domains HF have been identified as playing a key 
role in adverse events,'11213.1415 


The first domain to adopt a comprehensive HF awareness program was 
aviation, which introduced a training for aircrew entitled ‘Crew Resource 
Management’ (CRM) in 1979 after a series of accidents in which HF were 
found to be the root cause.'® 

CRM can be characterized as‘a team format which makes optimum use of all 
available resources - time equipment, procedures and people - to enhance 
process safety and effectiveness. The key aspect of CRM is a system 
approach to safety culture, focusing on ‘threat and error management’ 
Rather than focusing on individual failure, CRM aims to identify system 
flaws and uses standardized communication tools to improve process 
effectiveness and safety. 

Following the implementation of CRM, a decrease in adverse events led to 
CRM becoming the present day aviation operational standard, despite the 
lack of sound scientific proof of the impact of CRM on aviation safety out- 
come.” Meta-analyses of the effectiveness of CRM in aviation show positive 
reactions of course participants, enhanced learning, and desired behavioral 
changes."® 

Also, other critical domains that implemented the CRM format report 
encouraging behavioural changes, but were also unable to provide 
conclusive evidence on process safety.'°2021 


The joint efforts of several aviation and maritime institutes (Dutch Mari- 
time Pilots’ Corporation, Finnish Maritime Administration, Norwegian Ship 
owners’ Association, SAS Flight Academy, Silja Line, Swedish Maritime Ad- 
ministration, Swedish Ship owners’ Association and The Swedish Club) re- 
sulted in the launch of Maritime Resource Management (MRM) in 1993.” 


To investigate the effect of MRM implementation, the Swedish Club non- 
profit mutual insurer decided to compile all their navigational claims from 
1998-2007. Star Cruises, which implemented MRM in the 90's, stood out. 
Despite operating in difficult and congested waters, and in sharp contrast 
to other merchant and cruise operators, Star Cruises had zero insurance 
claims for the eight ships that they operated in the ten year period. This 
performance has been maintained to date. While MRM training is currently 
available in over 30 countries worldwide, no uniform standard for maritime 
HF training has been agreed upon. 


In the 90's CRM found its way into healthcare. As in the maritime environ- 
ment a medical CRM training standard is still lacking, resulting in CRM- 
initiatives that vary in content, concept and quality of training. This variety 
is a challenging basis for comparing studies on healthcare CRM. While the 
parallels between the critical processes in aviation and medicine suggest 
that a well-adapted medical CRM format may have potential to improve 
patient safety, research efforts into the effects of CRM on professional beha- 
viour and patient outcome yield an unsurprising myriad of results. 

In their 2014 meta-analysis O'Dea et al. found that CRM training had large 
effects on participants’ knowledge and behaviour, and a positive impact on 
teamwork in healthcare.” 

After reviewing six studies Boet et al. concluded that CRM skills learned 
at the simulation centre are transferred to clinical settings, and the 
acquired CRM skills may translate to improved patient outcomes, includinga 
decrease in mortality.” 

In alarge controlled study (74 facilities), Neilly et al. found that their Medical 
Team Training program was associated with lower surgical mortality.” 
Kemperetalrecently performed a robust paired controlled before-after study 
in six ICU’s with one pretest and two post-test measurements after 3 and 12 
months. In contrast to the aforementioned studies, it was concluded that 
their CRM implementation format did not change professional behaviour or 
patient outcomes. Whether these results were influenced by the relatively 
short follow-up period or the limited sample size remains unclear.” 


Most studies do agree on the need for improved standardization of methods 
and measures, greater precision in outcome assessment, and more robust 
research design. 


Thesis 

This thesis reviews the existing literature on Human Factors awareness 
training, provides an open source safety climate assessment tool for the 
Netherlands, and reports on the implementation of Crew Resource 
Management in highly critical clinical environments. 
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1.3 Outline 


Medical care frequently results in unintentional harm to patients. In 
chapter one we discuss patient safety from a Human Factors perspective, 
and introduce the concept of Crew Resource Management (CRM). 


Based on the Safety Attitudes Questionnaire (SAQ) benchmarked by Sexton 
et al. we developed and validated the SAQ-NL, an open-source instrument 
to measure safety climate for the Dutch clinical setting, The results are 
discussed in chapter two. 


Next, we adapted the aviation-derived CRM Human Factors awareness 
training curriculum to the clinical environment and conceived a three- 
phased structure for CRM interventions. Both instruments are discussed in 
chapter three. 


To determine whether implementation of CRM impacts outcome in 
critically ill patients and safety climate we conducted a study in the 
Intensive Care Unit to determine the effects of our CRM-intervention on 
ICU complication rates, ICU and hospital length of stay, standardized 
mortality ratio and safety climate (measured using the SAQ-NL). The results 
are described in chapter four. 


Chapter five describes the effects of a CRM-intervention in the Trauma 
Room setting of an Emergency Department (ED). Outcomes were safety 
climate (again measured using the SAQ-NL), ED and hospital length-of-stay, 
and 48-hour crude mortality of trauma patients. 


Experience from CRM integration projects in 19 high-risk clinical depart- 
ments has helped us identify several key success factors as well as threats 
to success that may be of use to future clinical CRM initiatives. Chapter six 
provides a longitudinal multicenter study into the impact of CRM on safety 


climate, and identifies factors for successful implementation insight on an 
operational as well as managerial level. 


Finally, chapter seven summarizes the results of the studies and discusses 
current issues and future perspectives. 
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Chapter 2 


Validation of the Dutch language 
version of the Safety Attitudes 
Questionnaire (SAQ-NL). 


MHTM Haerkens 
W van Leeuwen 
JB Sexton 

P Pickkers 

JG van der Hoeven 


BMC Health Services Research 2016; 16:385 


Abstract 


Background 

As the first objective of caring for patients is to do no harm, patient safety 
is a priority in delivering clinical care. An essential component of safe care 
in a clinical department is its safety climate. Safety climate correlates with 
safety-specific behaviour, injury rates, and accidents. Safety climate in 
healthcare can be assessed by the Safety Attitudes Questionnaire (SAQ), 
which provides insight by scoring six dimensions: Teamwork Climate, Job 
Satisfaction, Safety Climate, Stress Recognition, Working Conditions and 
Percep-tions of Management. 

The objective of this study was to assess the psychometric properties of the 
Dutch language version of the SAQ in a variety of clinical departments in 
Dutch hospitals. 


Methods 

The Dutch version (SAQ-NL) of the SAQ was back translated, and analyzed 
for semantic characteristics and content. From October 2010 to November 
2015 SAQ-NL surveys were carried out in 17 departments in two univer- 
sity and seven large non-university teaching hospitals in the Netherlands, 
prior to a Crew Resource Management human factors intervention. 
Statistical analyses were used to examine response patterns, mean scores, 
correlations, internal consistency reliability and model fit. Cronbach's a's 
and inter-item correlations were calculated to examine internal consistency 
reliability. 


Results 

1314 completed questionnaires were returned from 2113 administered to 
health care workers, resulting in a response rate of 62%. Confirmatory Factor 
Analysis revealed the 6-factor structure fit the data adequately. Response 
patterns were similar for professional positions, departments, physicians and 
nurses, and university and non-university teaching hospitals. The SAQ-NL 
showed strong internal consistency (a = .87). Exploratory analysis revealed 
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differences in scores on the SAQ dimensions when comparing different 
professional positions, when comparing physicians to nurses and when 
comparing university to non-university hospitals. 


Conclusions 

The SAQ-NL demonstrated good psychometric properties and is therefore 
a useful instrument to measure patient safety climate in Dutch clinical work 
settings. As removal of one item resulted in an increased reliability of the 
Working Conditions dimension, revision or deletion of this item should 
be considered. The results from this study provide researchers and practi- 
tioners with insight into safety climate in a variety of departments and 
functional positions in Dutch hospitals. 


Keywords 


Human Factors, Crew Resource Management (CRM), Safety Attitudes 
Questionnaire (SAQ), Dutch hospital setting 
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Background 


To err is human. As a result, everything that a human being devises, uses, or 
does is prone to error and failure. As this challenges the “First: do no harm” 
principle of healthcare’, it is imperative to assess the factors that impact 
patient safety. 

Patient safety is regarded by the National Patient Safety Foundation as the 
avoidance, prevention, and amelioration of adverse events or injuries stem- 
ming from the processes of healthcare? Identifying the key factors in safe 


clinical care is a challenging task. 


Evidence from non-clinical? and clinical*® critical environments suggests a 
positive relationship between safety culture, safety climate, and safety out- 
come. Safety culture is defined by the British Health & Safety Commission as 
“the product of individual and group values, attitudes, perceptions, compe- 
tencies, and patterns of behavior that determine the commitment to, and 
the style and proficiency of, an organization's safety management.’ From an 
anthropological standpoint, “safety culture” is only measurable by careful, 
long-term observations. Therefore, in daily clinical practice, it may be more 
appropriate to use the term “safety climate”, which generally refers to the 
measurable components of safety culture such as management behaviors, 
safety systems, and employee perceptions of safety. 

Safety climate can be determined by the Safety Attitudes Questionnaire 
(SAQ), a validated healthcare derivative of the Cockpit Management 
Attitudes Questionnaire’? that has been adapted to various clinical 
settings.* "' The initial extended version consists of 60 items including 30 
core items that are identical in all clinical settings. The short form version 
includes only the 30 core items. 

Previous factor analysis identified factors covering six domains of the safety 
climate: Teamwork Climate (6 items) is the perceived quality of collaboration 
between personnel. Job Satisfaction (5 items) is defined as positivity about 
the work experience. Safety Climate (7 items) is the perception of a strong 
and proactive organizational commitment to safety. Stress Recognition 
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(4 items) is acknowledgement of how performance is influenced by 
stressors. Working Conditions (3 items) is the perceived quality of the work 
environment and logistical support (such as staffing and equipment). 
Perceptions Of Management (5 items) is the approval of managerial action."° 
SAQ responses are given on a 5-point Likert scale (1 = disagree strongly, 2 
= disagree slightly, 3 = neutral, 4 = agree slightly, 5 = agree strongly). Two 
items (items 2,11) are reversed scored.’ 

Although the SAQ has been utilized in safety research in the Dutch care 
setting® > '4, no open source Dutch language version of the SAQ has 
been published to date. One exception is the observational study on the 
content validity and internal consistency of a Dutch translation of the SAQ 
by Devriendt and colleagues which was published during the course of our 
study." Although good content validity (CVI = .83) and internal consistency 
(a = .90) were reported, the sample in the study was limited to, and con- 
ducted in, a single hospital in the culturally different context of Belgium.'° 
Furthermore, even though Belgium and the Netherlands are neighboring 
countries the Dutch language differs from the Belgian-Dutch language 
(Flemish), which is clearly visible in the Belgian-Dutch questions. Contrary 
to our study, no certified interpreters and/or native English speakers perfor- 
med the translation and the adapted Brislin protocol of forward and back 
translation was not used. 

The Dutch hospital system consists of three levels of hospitals: large 
university hospitals, medium size non-university training hospitals and 
smaller rural hospitals. 

The aim of the current study was to assess the psychometric properties of 
the Dutch language version of the SAQ (SAQ-NL) and provide insight into 
safety climate in a variety of departments and functional positions in Dutch 
hospitals. 
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Background 


Design and setting 

From October 1* 2010 to November 1* 2015 a cross-sectional survey was 
conducted in 17 departments in two university and seven non-university 
teaching hospitals in the Netherlands as part of an intervention study 
evaluating the impact of Crew Resource Management (CRM) — human 
factors awareness training. This study focuses on the baseline data gathe- 
red before the CRM-training. 

The clinical departments (and number of health care providers) that parti- 
cipated in this study included: two Intensive Care Units (ICU, n = 281), five 
Operating Rooms (OR, n = 648), two Cardiac Catheterization Labs (CCL, n = 
56), one Medium Care Unit (MCU, n = 33), three Emergency Rooms (ER, n = 
163), one Coronary Care Unit - Heart First Aid unit (CCU-HFA, n = 45), one 
Radiotherapy department (RTX, n = 12), one Department of Gastroentero- 
logy and Hepatology (DGH, n = 40) and one Pharmacy department (n = 36). 


The Safety Attitudes Questionnaire - NL 

It was decided to use the original 30-item version of the SAQ benchmarked 
by Sexton et al.'® '® containing identical questions for all clinical settings as 
the basis for the Dutch version because of its usability in multiple clinical 
environments, good psychometric properties and open source accessibi- 
lity. 

When introducing a foreign language questionnaire, potential semantic 
and cultural differences need to be taken into account. To determine 
semantic equivalence (the translated items have the same meaning as in 
the original) in the translated version the SAQ was translated from English 
to Dutch and back again by native speakers (of which one is a certified 
interpreter) following the adapted Brislin protocol.'”'’® The translated 
version was reviewed for semantic properties and content. A subject matter 
experts group, consisting of clinical faculty (n = 3), psychologists (n = 2) and 
human factors specialists (n = 3), analyzed clarity and appropriateness of 
wording and each item's meaning in the cultural setting of the Netherlands. 
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Data collection 

All professionals of each participating department received an invitation 
to fill out the SAQ-NL. The first five departments were issued a paper and 
pencil version, all participants in subsequent departments received a link 
to an online questionnaire. There was no significant difference between the 
groups associated with method of administration. 


Statistical analysis 

Frequency tables were generated to provide an overview of age categories, 
gender, professional positions, departments, department tenure, and hos- 
pital tenure of the responders. To provide an overview of response patterns, 
percentages for missing values (MV) were generated. Further analysis of 
MV was done by first recoding all MV to ‘0’ and all responses to ‘1: These 
recoded values were then aggregated to yield an overall response score. 

A univariate analysis of variance (ANOVA) was performed with the over- 
all response score as dependent variable and profession and department 
as independent variables to check for differences in the overall response 
score. Independent t-tests were applied to compare the overall response 
scores between university and non-university hospitals and between 
medical staff (attending physicians and residents) and support personnel 
(nurses, operating room assistants, and operating room assistants). Mean 
scores were calculated per item and then aggregated to yield a mean score 
per SAQ dimension. Furthermore, to provide an overview of percentages 
of participants that agreed or disagreed with an item, responses of 1 and 2 
on the 5-point scale were recoded as ‘disagree’ and responses 4 and 5 were 
recoded as ‘agree! 

Scale reliability analyses with all items and for each dimension separately 
resulted in a corrected item-total correlation and a Cronbach's a if an item is 
deleted for the dimension-scale. An overview of missing values, means and 
standard deviations, percentages agree and disagree, corrected item-total 
correlations, Cronbach's a's, and Cronbach's a's if an item is deleted were 
calculated. 
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Based on the results of the factor analysis as performed earlier’, a 
confirmatory factor analysis (CFA) was performed on participants who fully 
completed the instrument (n = 604). CFA was performed with analysis of 
moment structures (AMOS) software”. 

We deemed a successful model was that with a Goodness of Fit Index (GFI) 
> 0.9, a Comparative Fit Index close to 0.95% and a Root Mean Square 
Error of Approximation (RMSEA) < 0.08”. The x? statistic is also given (a poor 
measure of model fit of measurement, but included here for reasons of 
convention). 

The unrestricted model was based on the structure of the original data- 
base. We fit a six factor unrestricted CFA model that contained the 30 items 
retained in the previous study of Sexton et al'° that confirmed the SAQ's 
construct validity. 

Mean scores and standard deviations for each SAQ-NL dimension were 
calculated for professional positions, physicians (residents and attending 
physicians) vs. nurses, departments, and academic status separately. Note 
that the category ‘nurses’ consists of nurses, operating room technicians, 
and anaesthesiology technicians. To explore whether groups differed 
on mean scores, multivariate analysis of variance (MANOVA) was utilized 
to interpret the mean scores. Because SPSS removes all participants with 
missing values in any combination of more than one independent varia- 
ble, three separate MANOVA's were performed with professional position, 
physicians vs. nurses, and university status of the hospital as independent 
variables and the mean scores on each dimension as dependent variables. 
Because dependent variables were not highly correlated and because it 
is robust to many violations of MANOVA, Pillai’s trace was utilized as the 
MANOVA test statistic.” 

Since no a priori hypotheses were formulated, a post-hoc Bonferroni test 
was utilized to interpret significant findings when the independent variable 
consisted of more than two groups. Finally, a bivariate correlation analysis 
was done to provide an overview of relations between SAQ-NL dimen- 
sions. For the correlation analysis, Pearson's correlation was used with a 
two-tailed test of significance. 
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Because ofthe large statistical power due to large sample size, corresponding 
effect sizes are reported to interpret significant findings. The following cut- 
offs were used: small effect (Cohen's d = 0.2, n = 0.01), medium effect (d = 
0.5, n= 0.06), large effect (d = 0.8, n? = 0.14). Data was analysed using SPSS 
Statistics 22 (IBM Corp., Armonk, NY, USA). A p-value < 0.05 was considered 
to indicate significance. 


Results 


Demographics 

1314 of 2113 surveys were returned for a response rate of 62%. This final 
sample consisted of 623 nurses (47%), 239 attending physicians (18%), 90 
residents (6.8%) and 214 “category other” (16%). A total of 148 participants 
(11%) did not provide their position details. The university hospitals (n = 
2) employed 441 respondents, 873 respondents were employed by non- 
university teaching hospitals (n = 7). The database contained one outlier 
department with an exceptionally low response rate of 21%. 

Detailed demographic and professional characteristics of the responders 
are shown in Table 1. 


SAQ-NL factor structure and multi-level modeling 

The SAQ-NL with six factors and 30 items was used in all the administra- 
tions reported here. The 6-factor model fit the data well: x?(390) = 931.18, 
p < 0.001, GFI = 0.90, CFI = 0.91, and RMSEA = 0.05. Item loadings on 
respective factors appear in Table 4. 


SAQ-NL item characteristics 

The subject matter experts adjusted the items until they agreed on the 
appropriateness of the semantic characteristics and deemed the content 
sufficient and appropriate for measuring safety climate in hospitals. 
Missing values (MV) analysis revealed a range of 3.0 - 6.8% MV for the 
separate questions, see Table 4. ANOVA revealed no difference in MV for 
professional position, F(3, 1110) = 0.02, p = .996, or department, F(7, 1110) 


27 


= 1.23, p = 0.283. Independent t-tests revealed no difference in MV for 
university status, t(1283.83) = 1.83, p = 0.059, and physicians vs. nurses 
t(950) = 0.75, p = 0.452. Due to a technical error, item 16 (“this is a good 
place to work”) did not appear in the questionnaire initially and therefore 
resulted in a MV of 50%. 


SAQ-NL mean scores 

An overview of mean scores and standard deviations for comparison is 
provided in Table 2. Using Pillai’s trace, the overall MANOVA‘s revealed a 
medium effect of clinical position (n = 1159), V = 0.19, F(18, 3456) = 13.25, 
p <0.001, nee = 0.07, a large effect of physicians vs. nurses (n = 947), V = 0.14, 
F(6, 940) = 26.37, p < 0.001, n =0.14, and a small effect of academic status of 
the hospital (n = 1257), V = 0.03, F(6, 1250) = 6.65, p < 0.001, n =0.03, on the 
six SAQ-NL dimensions; Teamwork Climate, Safety Climate, Job Satisfaction, 
Stress Recognition, Perceptions of Management, and Working Conditions. 
Follow-up univariate ANOVA's revealed that there was an effect of profes- 
sional position on Teamwork Climate, F(3, 1155) = 49.08, p < 0.001, n = 
0.11, on Safety Climate, F(3, 1155) = 22.63, p < 0.001, n, = 0.06, on Job 
Satisfaction, F(3, 1155) = 23.69, p < 0.001, n = 0.06, on Perceptions of 
Management, F(3, 1155) = 2.95, p = .032, n = 0.01, and on Working 
Conditions, F(3, 1155) = 13.63, p < 0.001, Ne = 0.03. An overview of means 
and confidence intervals is provided in Figure 1. 

A post-hoc Bonferroni test revealed that attending physicians were more 
positive about Teamwork Climate than both residents, p < 0.001, d = 
0.35, and nurses, p < 0.001, d = 0.90. Residents were more positive about 
Teamwork Climate than nurses, p < 0.001, d = 0.59. For Safety Climate, 
attending physicians were more positive than residents, p = 0.008, d = 0.06, 
and nurses, p < 0.001, d= 0.64. Furthermore, nurses experienced lower Job 
Satisfaction than attending physicians, p < 0.001, d = 0.59, and residents, 
p = 0.001, d = 0.47. Finally, nurses were less positive about Working 
Conditions than attending physicians, p < 0.001, d = 0.42, and residents, 
p = 0.001, d = 0.46. 
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The follow-up univariate ANOVA's concerning physicians vs. nurses 
revealed that physicians were more positive about Teamwork Climate 
than nurses, F(1, 945) = 111.90, p < 0.001, n = 0.12. Physicians were more 
positive about Safety Climate than nurses, F(1, 945) = 60.43, p < 0.001, n, = 
0.06. Physicians experienced more Job Satisfaction than nurses, F(1, 945) = 
65.23, p < 0.001, Ae = 0.07. Physicians had higher Perceptions of Manage- 
ment than nurses, F(1, 945) = 4.73, p = 0.030, n, = 0.01. Finally, physicians 
were found to experience better Working Conditions than nurses, F(1, 945) 
= 30.12, p < 0.001, n = 0.04. An overview of means and confidence inter- 
vals is provided in Figure 2. 

Follow-up univariate ANOVA’s related to university status of the hospital 
revealed that university hospitals were more positive about Teamwork 
Climate than teaching hospitals, F(1, 1255) = 6.23, p = 0.013, n, = 0.01. 
Also, more Job Satisfaction was experienced in university hospitals than in 
teaching hospitals, F(1, 1255) = 7.28, p = 0.007, n = 0.01. Scores on Stress 
Recognition were lower in academic hospitals than in teaching hospitals, 
F(1, 1255) = 6.91, p= 0.009, n, = 0.01. In university hospitals, Perceptions 
of Management were higher than in teaching hospitals, F(1, 1255) = 33.54, 
p < 0.001, Ne = 0.03. Finally, university health care providers from hospitals 
were more positive about Working Conditions than teaching hospitals, F(1, 
1255) = 9.58, p = 0.002, Nes 0.01. An overview of means and confidence 
intervals is provided in Figure 3. 


Reliability and correlation analysis 

Reliability analysis of the SAQ-NL showed strong internal consistency, 
Cronbach's a = .87, see Table 4. For the Perceptions of Management and 
Working Conditions categories Cronbach's a's were below the .70 reliability 
threshold (.65 and .57, respectively) though. Interestingly, in spite of having 
no effect on overall SAQ-NL reliability, exclusion of item 29 would result in 
the Working Conditions dimension reliability increasing from .57 to .70. 
Teamwork Climate and Safety Climate were correlated at about .70. In 
addition, Stress Recognition was consistently negatively related to all other 
categories (see Table 3). 
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Discussion 


We developed and refined a Dutch language version of the SAQ and used 
it on a broad sample of hospital departments in the Netherlands. CFA 
confirmed the appropriateness of the proposed model and the resulting 
psychometric properties were good for this instrument. Internal consistency 
as well as correlations were similar to the results published by Sexton and 
colleagues (2006) in their validation study of the SAQ.'® 

Furthermore, reference data were reported for comparison purposes. In a 
pattern of results quite similar to what has been found in other translations 
of the SAQ'® %, the SAQ-NL was associated with significant unit-level 
variability, higher scores for physicians than non-physicians, and psycho- 
metrically valid scales. 

Explorative analyses of the data revealed two interesting findings. First, the 
robust finding that physicians score higher in 5 out of 6 SAQ-NL domains 
than nurses is consistent with previous research.” This represents a different 
perception of the safety climate within clinical teams, a factor that should 
be taken into account during human factors awareness training. Second, 
university hospitals were found to be slightly more positive about safety 
climate than non-university teaching hospitals. A possible explanation 
might be the lower clinical production pressure perceived in the academic 
setting, as well as a teaching environment with more emphasis on super- 
vision. However, university hospitals scored slightly lower in stress recog- 
nition. We can offer no explanation for this finding. Several studies find 
that the SAQ-factor Stress Recognition has problems regarding construct 
validity and that it does not vary significantly between organizational 
units? 


Strengths 

The first strength of the present study is the broad spectrum of partici- 
pating hospitals, departments and professionals resulting in a sample that 
could be considered a representative cross section of acute and critical care 
departments in the Dutch clinical healthcare setting. In addition, the large 
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sample size resulted in sufficient representation of professionals in the 
categories utilized in this study. Thirdly, as this study provides an open 
source Dutch translation of the SAQ short form, it may serve as a basis for 
future research. This would allow for better comparison of future investiga- 
tions into safety climate in hospital departments in the Netherlands. 


Limitations 

The most important limitation of the present study is the fact that hospital 
departments were not a random sample. The SAQ-NL was determined 
in units that were to receive human factors training, and it is therefore 
possible that these non-random units had safety culture norms that were 
not representative. One could argue that the fact that they signed up for 
human factors training could be the result of priority given to safety climate 
resulting in a higher safety culture norm than expected, or the opposite, 
that these departments wished to participate because of perceived 
problems with safety. A brief comparison of our overall means to other 
samples suggests that the latter was not the case. Nevertheless this would 
not impact the psychometric results, which ranged from adequate to good. 
Second, in spite of our efforts to include as many different departments and 
clinical specialties as possible, we recognize this study cannot encompass 
the total clinical spectrum. We therefore encourage further research 
covering even more clinical specialties inside and outside of inpatient 
settings. 

Third, item 16 (“this is a good place to work”) did not appear in the 
questionnaire initially and therefore resulted in a MV of 50%. However, the 
large sample size limits the impact of this omission. 

Finally, this study period covered 5 years. Possible effects of general 
changes in perceptions of clinical safety climate during this timeframe 
cannot be excluded. Nevertheless, results from the first two years compared 
to the last two years did not yield significant differences (data not shown), 
indicating that this is not likely to be an issue. 
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Perceived safety climate is associated with safety outcomes in hospital 
settings.” Therefore, determination of safety climate is of clinical relevance. 
The SAQ-NL in its present form shows promise to be a benchmarked 
tool for future research into patient safety. Exclusion of item 29 “All the 
necessary information for diagnostic and therapeutic decisions is routinely 
available to me” would result in an increase of Working Conditions dimen- 
sion reliability (from .57 to .70). Even though this would not impact overall 
SAQ-NL reliability, adapting, deleting, or at the very least, monitoring this 
item is something to consider in future research that utilizes the SAQ-NL. 
After this adjustment psychometric properties should be reassessed in a 
randomly selected sample and hospitals and departments prior to more 
widespread use in Dutch hospital settings. 


Conclusions 


We assessed the psychometric properties of the Dutch language version of 
the SAQ, the SAQ-NL, and provided insight into safety climate in a variety of 
clinical departments in Dutch hospitals. The SAQ-NL is a reliable instrument 
to measure safety climate in the Dutch hospital setting. Further research 
is needed to validate the SAQ-NL as a monitoring tool for pre-and-post 
administration of the impact of interventions related to safety climate. 
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List of abbreviations 


AMOS Analysis of MOment Structures 

ANOVA Univariate analysis of variance 

CFA Confirmatory Factor Analysis 

SAQ Safety Attitudes Questionnaire 

SAQ-NL Dutch language version of the SAQ 
CRM Crew Resource Management 

ICU Intensive Care Unit 

OR Operating Room 

CCL Cardiac Catheterization Lab 

MCU Medium Care Unit 

ER Emergency Room 

CCU-HFA Coronary Care Unit - Heart First Aid unit 
RTX Radiotherapy 

DGH Department of Gastroenterology and Hepatology 
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Figures 





nurses “residents "att. physicians 


Figure 1 
SAQ Means for Professional Position. Overview of mean scores and 95% 


Confidence Intervals. 
Att. physicians = attending physicians; Perc. of Management = Perceptions 


of Management. 
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nurses physicians 


Figure 2 
SAQ Means for Physicians versus Nurses. Overview of mean scores and 95% 


Confidence Intervals. 
Perc. of Management = Perceptions of Management. 


38 





academic ™ non-academic 


Figure 3 

SAQ Means for Academic Status. Overview of mean scores and 95% Confi- 
dence Intervals. 

Perc. of Management = Perceptions of Management. 
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Table 1 
Frequency Table for Participant Demographic Variables, Departments, and Tenure 





a 








Age Gender Position Department Tenure at Tenure at 
department? hospital? 
Cat. Freq. (%) Cat. Freq. (%) Cat. Freq. (%) Freq. (%) Cat. Freq (%) Cat. Freq. (%) 
< 20 25 (1.9) Male 400 (30.4) Nurse” 623 (47.4) Intensive care 281 (21.4) <1 122 (9.3) <1 86 (6.5) 
21-30 210 (16.0) Femal 813 (61.9) Resident 90 (6.8) Operating room 648 (49.3) 1-5 382 (29.1) 1-5 304 (23.1) 
e 
31-40 322 (24.5) Att. 239 (18.2) CCL 56 (4.3) 6- 243 (18.5) 6- 237 (18.0) 
physician 10 10 
41-50 365 (27.8) Other 214 (16.3) Medium care 33 (2.5) >10 476 (36.2) >10 596 (45.4) 
> 50 304 (23.1) Emergency 163 (12.4) 
room 
CCU - HFA 45 (3.4) 
Radiotherapy 12 (0.9) 
DGH 40 (3.0) 
Pharmacy 36 (2.7) 
Missing 88 (6.7) 101 (7.7) 148 (11.3) 0 91 (6.9) 91 (6.9) 





Note. N = 1314; Cat. = Category; Freq. = Frequency; Att. physician = Attending physician; CCL = Cardiac Catheterization Lab; CCU-HFA = 
Coronary Care Unit - Heart First Aid unit; DGH = Department of Gastroenterology and Hepatology; è Age and tenure in years; ° Nurse 
category consists of nurses, operating room technicians, and anaesthesiology technicians. 
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Table 2 


SAQ-NL Means and (Standard Deviations) per Professional Position and Department 





Teamwork Safety Job Stress Perc. of Working 
Climate Climate Satisfaction Recognition Management Conditions 
Position 
Nurse 3.49 3.37 (0.52)* 3.50 2.97 (0.70) 2.89 (0.60) 3.23 
(0.55)* (0.64)* (0.61)* 
Resident 3.79 3.50 (0.50)* 3.77 2.92 (0.72) 3.03 (0.53) 3.49 
(0.47)* (0.51)* (0.53)* 
Attending 3.96 3.72 (0.58)* 3.87 3.07 (0.74) 2.96 (0.62) 3.49 
physician (0.49)* (0.62)* (0.63)* 
Other 3.47 (0.64) 3.52 (0.64) 3.75 (0.70) 2.98 (0.82) 3.01 (0.62) 3.24 (0.71) 
Physician vs Nurse 
Physicians 3.91 3.66 (0.57)* 3.85 3.03 (0.73) 2.98(0.60) 3.49 
(0.49)* (0.59)* (0.60)* 
Nurses 3.49 3.37 (0.52)* 3.50 2.97 (0.70) 2.89 (0.60) 3.23 
(0.55)* (0.64)* (0.61)* 
Department 
Intensive care 3.74 (0.47) 3.50 (0.44) 3.72 (0.49) 2.95 (0.60) 3.07 (0.51) 3.36 (0.52) 
Operating room 3.48 (0.60) 3.41 (0.61) 3.48 (0.71) 3.00 (0.74) 2.84 (0.62) 3.28 (0.65) 
CCL 3.76 (0.53) 3.71 (0.53) 4.01 (0.53) 2.79(0.72) 3.39 (0.50) 3.07 (0.71) 
Medium care 3.68 (0.37) 3.55 (0.37) 3.70 (0.33) 2.58 (0.48) 3.14 (0.44) 3.36 (0.43) 
Emergency room 3.66 (0.54) 3.41 (0.52) 3.81 (0.59) 3.03 (0.74) 2.92 (0.61) 3.32 (0.57) 
CCU - HFA 3.74 (0.63) 3.57 (0.74) 3.87 (0.58) 3.08 (0.93) 2.70 (0.64) 3.44 (0.77) 
Radiotherapy 3.72 (0.56) 3.85 (0.48) 3.97 (0.37) 3.06 (0.75) 3.51 (0.39) 3.25 (0.75) 
DGH 4.01 (0.52) 3.77 (0.48) 4.05 (0.46) 3.07 (0.74) 2.86 (0.56) 3.15 (0.73) 
Pharmacy 3.34 (0.68) 3.59 (0.74) 3.88 (0.68) 3.16 (0.94) 2.96 (0.60) 3.36 (0.97) 
Academic status 
Academic 3.65 3.50 (0.50) 3.71 2.91 (0.65)* 3.07 (0.49) 3.37 
(0.47) (0.47)* (0.56)* 
Non-academic 3.57 3.45 (0.60) 3.61 3.02 (0.76)* 2.87 (0.65) 3.26 
(0.63)* (0.73) (0.67) 


Note. N = 1314; * Between group differences at p < 0.05; CCL = Cardiac 
Catheterization Lab; CCU - HFA = Coronary Care Unit - Heart First Aid unit; DGH = 
Department of Gastroenterology and Hepatology; Perc. of Management = Perceptions 


of Management. 
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Table 3 


SAQ-NL Dimension Means and (Standard Deviations), Correlations, and Cronbach’s a's 








Mean (SD) 1. 2. 3. 4. 5. 6. 
1. Teamwork Climate 3.60 (0.58) .76 
2. Safety Climate 3.47 (0.57) .73 77 
3. Job Satisfaction 3.65 (0.65) .56 54 ‚84 
4. Stress Recognition 2.99 (0.73) -.14 -.15 -.15 ‚69 
5. Perc. of Management 3.01 (0.66) .33 „36 „40 -.17 ‚65 
6. Working Conditions 3.13 (0.56) .47 48 40 -.18 35 57 





Note. N = 1314; Perc. of Management = Perceptions of Management; All correlations 
are significant at the p < 0.01 level; Cronbach's a's appear in boldface on the diagonal. 
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Table 4 


SAQ Items, Translations, Response Rates, Means, Standard Deviations, 


Factor Loading, and Reliability Characteristics 





Missing Mean Disagree/ Corrected CFA a if item 

Dimensions and questions (9%) (SD) agree item-total factor deleted 
(%) correlation loadings (dimension) 

Total scale a = .87 
Teamwork Climate 76 
1. Nurse input is well received in this clinical area. 37 3.76 7.0/65.8 49 78 73 
De inbreng van verpleegkundigen wordt op mijn unit op prijs gesteld. (0.84) 
2. In this clinical area, it is difficult to speak up if | perceive a problem with patient 3.5 2.53 52.4 / 14.8 A0 78 75 
care. (0.93) 
Op mijn unit is het moeilijk om het uit te spreken als ik merk dat er een probleem 
is met de patiëntenzorg. 
3. Disagreements in this clinical area are resolved appropriately (i.e not who is 34 3.26 16.6 / 40.1 65 1.12 71 
right, but what is best for the patient). (0.87) 
Meningsverschillen op mijn afdeling worden op een goede manier opgelost 
(d.w.z. niet wie heeft er gelijk, maar wat is het beste voor de patiënt). 
4. | have the support | need from other personnel to care for patients. 37 3.65 8.2/59.5 57 88 71 
Ik krijg de ondersteuning die ik nodig heb van staf-artsen om voor patiënten te (0.84) 
kunnen zorgen. 
5. It is easy for personnel here to ask questions when there is something that they 3.3 3.96 3.8/76.6 57 86 72 
do not understand. (0.76) 
Medewerkers op mijn unit kunnen gemakkelijk vragen stellen als er iets is dat ze 
niet begrijpen. 
6. The physicians and nurses here work together as a well-coordinated team. 44 3.48 11.9 / 48.1 53 1.00 72 
De artsen en de rest van het team hebben hier een goede samenwerking. (0.89) 
Safety Climate 7 
7. | would feel safe being treated here as a patient. 3.6 3.78 5.4/67.2 56 „80 73 
Als ik hier als patiënt zou worden behandeld, zou ik me veilig voelen. (0.79) 
8. Medical errors are handled appropriately in this clinical area. 5.0 3.38 11.3/ 43.0 56 „90 73 
Medische fouten worden goed afgehandeld op de afdeling. (0.84) 
9. | know the proper channels to direct questions regarding patient safety in this 40 3.63 11.6 / 60.8 37 62 76 
Clinical area. (0.94) 
Ik weet aan wie ik vragen kan stellen als het gaat om de patiëntveiligheid op de 
afdeling waar ik werk, 
10. | receive appropriate feedback about my performance. 3.6 3.29 18.0/42.5 45 74 74 
Ik krijg goede feedback op mijn functioneren. (0.92) 
11, In this clinical area, it is difficult to discuss errors. 3.0 2.53 52.7 / 15.4 52 92 73 
Op de unit waar ik werk is het lastig om fouten te bespreken. (0.93) 
12, | am encouraged by my colleagues to report any patient safety concerns | 34 3.43 12.3 / 46.8 45 „70 74 
may have. (0.86) 
Ik word door mijn collega's aangemoedigd al mijn bedenkingen wat 
patiëntveiligheid betreft te melden. 
13, The culture in this clinical area makes it easy to learn from the errors of 33 3.31 15.4 / 42.2 59 1.00 71 
others. (0.86) 


De cultuur op mijn unit maakt het makkelijk om van fouten van anderen te leren. 
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Job Satisfaction 84 


14. I like my job. 53 4.09 3.3/78.3 56 95 „80 
Ik ben enthousiast over mijn baan. (0.78) 

15, Working here is like being part of a large family. 65 2.99 22.7/25.7 48 1.01 82 
Het werken in dit ziekenhuis voelt als deel uit maken van een grote familie. (0.91) 

16. This is a good place to work. 50.0 3.80 46/351 57 99 „80 
Dit ziekenhuis is een goede plek om te werken. (0.91) 

17. Lam proud to work in this clinical area. 5.6 3.87 6.6 / 69.4 63 1.20 77 
Ik ben trots dit ik in dit ziekenhuis werk. (0.88) 

18. Morale in this clinical area is high. 48 3.53 14.2/55.3 58 1.00 83 
Het moreel op deze afdeling is hoog. (0.98) 

Stress Recognition „69 
19. When my workload becomes excessive, my performance is impaired. 40 3.23 26.3 / 41.7 -15 1.16 62 
Wanneer mijn werkdruk te hoog wordt, dan lijdt mijn functioneren daaronder. (1.05) 

20. | am less effective at work when fatigued. 68 2.89 34.6 / 26.1 -06 99 64 
Als ik vermoeid ben dan verricht ik routinetaken minder goed. (0.96) 

21, I am more likely to make errors in tense or hostile situations. 38 2.99 32.2 / 30.4 „04 1.10 63 
Ik ben meer geneigd om fouten te maken in een gespannen of bedreigende (1.00) 

situatie. 

22. Fatigue impairs my performance during emergency situations. 49 282 36.6 / 24.1 -09 1.00 63 
Vermoeidheid hindert mijn functioneren tijdens acute situaties. (0.96) 

Perceptions of Management 65 
23, Management supports my daily efforts. 53 2.70 34.0 / 13.9 42 2.04 56 
Het ziekenhuismanagement helpt me bij mijn dagelijkse bezigheden. (0.87) 

24, Management doesn't knowingly compromise patient safety. 54 3.54 10.0/52.5 39 1.59 62 
Het ziekenhuismanagement brengt de veiligheid van de patiénten niet bewust in (0.88) 

gevaar. 

25. Problem personnel are dealt with constructively by our unit / hospital 58 2.63 35.8/9.2 44 1.59 61 
management. (0.82) 

Dit ziekenhuis gaat constructief om met minder goed functionerend personeel. 

26. | get adequate, timely info about events that might affect my work, from unit / 47 2.93 29.3/ 27.5 53 2.53 53 
hospital management. (0.94) 

Ik krijg voldoende, tijdige informatie over gebeurtenissen in het ziekenhuis die 

invloed kunnen hebben op mijn werk. 

27. The levels of staffing in this clinical area are sufficient to handle the number of 52 2.89 45/314 24 1.00 67 
patients. (1.08) 

We hebben genoeg personeel om de werklast aan te kunnen. 

Working Conditions 57 
28. This hospital does a good job of training new personnel. 5.3 3.20 14.5 / 32.6 AT „93 „33 
Dit ziekenhuis is goed in het trainen van nieuw personeel. (0.82) 

29. All the necessary information for diagnostic and therapeutic decisions is 5.5 3.18 18.7/34.3 36 36 70 
routinely available to me. (0.85) 

Ik beschik steeds over alle informatie die nodig is voor diagnostische en 

therapeutische beslissingen. 

30. Trainees in my discipline are adequately supervised. 46 3.50 13.5/55.1 43 1.00 „31 
Degenen die opgeleid worden in mijn discipline krijgen voldoende begeleiding. (0.90) 


Note. N = 1314; Total scale and dimension Cronbach's a's appear in boldface; Dutch translated items appear in italics underneath original 
item. Items 2 and 11 are reverse scored. 
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Abstract 


Intensive care frequently results in unintentional harm to patients and 
Statistics don’t seem to improve. The ICU environment is especially 
unforgiving for mistakes due to the multidisciplinary, time-critical nature of 
care and vulnerability of the patients. 

Human Factors account for the majority of adverse events and a sound 
safety climate is therefore essential. This article reviews the existing 
literature on aviation-derived training called Crew Resource Management 
(CRM) and discusses its application in critical care medicine. CRM focuses 
on teamwork, threat and error management and blame free discussion of 
human mistakes. Though evidence is still scarce, the authors consider CRM 
to be a promising tool for culture change in the ICU setting, if supported by 
leadership and well-designed follow-up. 
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Introduction 


Despite modern equipment, continuing research and excellent training 
facilities our western health care system has a serious safety problem. It is 
estimated that out of all hospital admissions 2.9% to 16.6% suffer uninten- 
tional harm and in the United States alone up to 100,000 people may die 
as a result of medical errors.' Data from the Netherlands (2004) seem to 
support this with an annual number of 30.000 patients suffering preven- 
table harm including approximately 1.735 avoidable deaths’. The financial 
cost of avoidable adverse events was estimated to be 1% of the hospital 
total budget. 


These alarming reports resulted internationally in increased pressure to 
improve patient safety. In line with the current safety paradigm that 
limiting human variability in otherwise safe systems will lead to safer care“, 
this resulted in more stringent procedural guidelines. Checklists, time- 
outs and safety management systems were subsequently implemented. 
Unfortunately, current efforts have not eliminated human error? and as 
expected matters are worse in the ICU environment.®’® Patients in the ICU 
frequently suffer from severe, multiple-system disorders that require more 
testing, monitoring, invasive treatment and intravenous medications than 
in any other hospital department.’ Errors in this setting have a greater 
impact due to the increased vulnerability of the patient. The sheer number 
of available data essential to make a single decision is daunting. Further- 
more, ‘rogue’ data (irrelevant but abnormal e.g. a high glucose value) 
unrelated to the true problem can cause a change or loss in focus on the 
“real” problem. This puts great pressure on multidisciplinary planning and 
decision making and combined with the time-critical aspects of the ICU 
environment increases patient risk. 


Identifying the key factors in safe critical care is a challenging task. Human 
factors appear to play an important role. Less often organizational and 
technical factors are involved. This is consistent with publications from 
other critical industries." 
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Safety Climate 


If safe critical care relies on the interdisciplinary performance of a care team 
as muchas on individual expertise, it makes sense to establish a sound safety 
culture as the basis of improving patient outcomes.!? From an anthropolo- 
gical standpoint, “safety culture” is only measurable by careful, long-term 
observations. Therefore, in the evidence-driven medical world it may be 
more appropriate to use the term “safety climate”, which can be assessed 
by questionnaires. The Safety Attitudes Questionnaire (SAQ) is widely 
regarded to be valid, reliable, psychometrically sound and responsive to 
interventions.'? Evidence from other critical industries suggests that 
“safety climate” correlates with unsafe and safety-specific behavior, 
injury rates and accidents.'*'® Likewise, in the medical literature the “safety 
climate” of a hospital unit is considered one of the main contributing 
factors to a better quality of care. How could we improve this safety 
climate? 


The majority of current interventions focus on implementing safety tools 
such as event-reporting systems, quality and safety dashboards, evidence- 
based guidelines and checklists. Even though the results ofa comprehensive 
unit-based safety program (CUSP) are promising!®'®, introducing more 
stringent rules potentially increases the gap between procedure and 
practice. Therefore, the question remains if these tools can be truly effec- 
tive in the traditional hospital climate, where highly trained professionals 
tend to focus more on individual performance than team effectiveness.'® 
Moreover, the typical culture in which junior members of the ICU staff 
should not question the decisions made by senior members adds to the 
challenge." 
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Parallels 


We can definitely learn from methods developed in other critical 
industries, despite the perceived procedural differences with health care.”°" 
One example is the professional civilian or military aviation industry. Until 
1977, the aviation industry could be classified as a professional-centered, 
hierarchic working environment. This all changed with the Tenerife disaster. 
A KLM Boeing 747 at take-off crashed into Pan Am flight 736 still taxi- 
ing on the Los Rodeos airport runway. The accident investigation report 
(backed by objective cockpit voice and flight data recorder information) 
later revealed that human factors contributed to the deadliest mishap in 
aviation history, which claimed 583 lives. Ineffective radio communication 
with Air Traffic Control due to non-standard terminology and language 
barrier issues led to misinterpretation of the actual situation and a 
premature take-off decision of the KLM captain. The steep authority- 
gradient in the Dutch cockpit prevented the crew from challenging the 
captain's decision. As part of the solution to prevent this from happening 
ever again, a compulsory Human Factor training for all aircrew personnel 
was advised after follow-up research. This annual training, called Crew 
Resource Management (CRM), was developed in 1979 in a workshop 
sponsored by NASA.” CRM has meant a huge leap forward in improving 
aircrew team performance in civilian as well as in military aviation. The 
focus of CRM is on threat and error management and early identification 
with blame free countering of human mistakes. CRM training is now 
mandatory for professional aircrew in Europe and the United States. 


How does this fit in the intensive care environment? In the ICU there is 
inherently much emphasis on technical skills and not on communication, 
teamwork and leadership. These latter skills are rarely deliberately taught 
or sought after from applicants”, resulting in critical care practitioners 
being relatively unprepared to meet the demands of the increasingly com- 
plex ICU environment.’ This is the basis for suboptimal coordination of 
multidisciplinary care and the resultant high number of ICU errors.” The 
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traditional critical care environment has a tendency to focus on the perfor- 
mance of a particular practitioner rather than on the system of care. 


In aviation, nontechnical skills, a blame-free environment and Team 
Situational Awareness (SA) are considered CRM core competencies that 
require specific and focused training.” Team SA is defined as the ability 
to identify, process, and comprehend the critical elements of information 
about what is happening to the team with regards to the mission. Team 
SA is considered to be the basis for effective decision making in critical 
environments and a core competence for any professional team. The 
archetypical medical specialist's personality (highly motivated, A-type, 
control freak) helps to create an environment in which a junior team 
member could feel inhibited to offer input in a senior team with “vertical” 
leadership. This impacts Team SA, posing a threat to process safety, and 
thus patient safety.” 


Where is the evidence? 


ICUs with a “team-oriented culture” have shorter lengths of stay, lower 
nursing turnover, higher quality of care and can better meet family 
members’ needs.” As discussed earlier, general information on a depart- 
ment’s safety climate may be obtained by questionnaires and reviews 
of complication data.'f Objective team performance data in regards to 
specific adverse events is hard to obtain. Although the patient monitor 
and ventilator store data, the process by which decisions are made is only 
available afterwards in the form of doctor's and nurse’s notes. Video 
monitoring with voice recording is not widely available for debriefing 
purposes. This limits the visibility of the role of Human Factors in peer 
reviews and morbidity and mortality conferences. The effect of natio- 
nal rules and regulations cannot be overestimated. In the United States, 
according to the Health Insurance Portability and Accountability Act 
(HIPAA), the simple concept of video recording a trauma resuscitation 
requires review by a lawyer and, according to the Joint Commission in the 


50 


United States, patient permission and is, thus, not a widely accepted prac- 
tice. To date, no evidence is available from the ICU environment that CRM 
training improves patient safety. Notably, most team training evaluations 
have been conducted in the military and aviation environment.” These 
results look promising, and recent evidence also shows a positive effect 
of team training in the operating room.”®”? Neily et al. analyzed surgical 
mortality data from 108 Veterans Affairs Hospitals and showed that a 
Medical Team Training program resulted in significant reduction in 
surgical mortality rates.” Unfortunately, results from other authors are less 
favourable. Even though non-technical skills, attitudes and teamwork 
climate seem to improve, no significant effect on operating time or length 
of hospital stay (LOS) was found.” Considerable cultural resistance to 
adoption is encountered, particularly among medical staff. Debriefing and 
challenging authority seemed more difficult to adopt than other parts of 
the training.” 


Changing the climate: implementation 


Crew Resource Management training for Royal Netherlands Air Force 
(RNLAF) aircrew is a 2-day full-time interdisciplinary training. The training 
syllabus consists of lectures in cognitive psychology and multiple inter- 
active sessions using realistic data. 


Key subjects in the CRM-syllabus are: 

e Situational Awareness and recognition of adverse situations 
e Human errors and nonpunitive response 

* Communication and crosscheck techniques 

e Give and receive performance feedback 

+ Management of stress, workload and fatigue 

e Creating and maintaining team structure and climate 

e Leadership 

+ Risk management and decision-making 
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Any CRM-training has to meet Federal Aviation Authority (FAA) or Joint 
Aviation Authority (JAR) regulations. Not only do they define the various 
subjects but also the extent to which each subject should be discussed and 
set limits for refresher training. This standardization is a major contributing 
factor to the success of CRM. 


Medical CRM-training has no international standard yet. Medical Human 
Factors awareness training initiatives may vary in curriculum, duration, 
intensity and follow-up support. The U.S. Department of Defense's Patient 
Safety Program developed TeamSTEPPS, an evidence-based teamwork 
system, in collaboration with the Agency for Healthcare Research and 
Quality (AHRQ). TeamSTEPPS has been implemented in a variety of 
clinical settings and shown team performance improvement in pediatric 
and surgical ICUs. Still, any hospital department deciding that CRM is the 
way forward to improve patient safety should realize that it is not just a 
single shot training investment (which can be very effective in itself)? but 
part of a culture intervention. There will be understandable reluctance in 
the medical community to accept the necessity of a CRM-culture interven- 
tion in their professional environment. Even though further studies are 
needed to define the optimal training package”, some basic guidelines 
may be given. 


1. CRM training. The goal of this training is creating awareness of the 
human factors that influence team performance. We suggest a 2-day full 
time training containing the key subjects of aviation CRM as discussed 
above. Because the multidisciplinary ICU environment requires a different 
non-technical skillset than a cockpit, medical CRM training should be 
tailored to the specific department's environment. This is where some 
current training initiatives fall short. Tailoring CRM-training to the specific 
needs of an ICU requires insight in the specific clinical processes and 
culture? ICU professionals have no tradition in briefing and debriefing 
techniques and performance feedback. 
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ICU-CRM training should therefore emphasize: 

- Briefing and debriefing skills using exercises and actual ICU video foot- 
age 

e _ Effective use of checklists 

e Identifying team roles 

* Promoting structure, reduction of ineffective communication 

e Performance feedback as an essential requirement in CRM. This starts 
with careful consideration of timing and relevance of the message, 
followed by three levels of performance feedback. The first level 
requires the team member to formulate the message short, clear 
and non-blaming (“Doctor, I’m not sure we did all the checks...”). The 
second level contains a key word that has a defined value (“Doctor, 
I'm not comfortable with that decision...”). If this feedback is ignored, 
and the situation is considered unacceptable the last resort could be a 
request to “stop the procedure”. 


One CRM-tool used at the Mayo Clinic to help facilitate accurate communi- 
cation in stressful medical situations is based on the work by Patterson et 
al. Medical personnel are taught how to communicate without creating 
conflict or in the face of apparent conflict. This model has proven useful 
while multiple other endeavors have been created and implemented to 
foster the ultimate safe environment. 

We are currently developing evidence-based requirements for a national 
ICU-CRM training curriculum in the Netherlands. The basis is a 2-day CRM- 
training using lectures, video-feedback and interactive exercises. This 
training is followed by a 1-year implementation phase in which a core 
group of department professionals is coached by a CRM trainer. Results will 
be published in the near future. 


2. CRM implementation. To be successful, the culture change should be 
supported by additional measures. A core group of ICU-professionals 
should receive extra coaching during the year after the training to be able 
to integrate and develop the new way of professional interaction within 
the ICU-department. Even though CRM relies on intrinsic motivation to be 
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effective, the department leadership needs to clarify to all staff beforehand 
that CRM is not a noncommittal system but will serve as a yardstick for 
professional evaluation too. This requires leadership by example. 

3. CRM and simulation. The effect of CRM-based culture change is reinforced 
by the use of scenario-based team training exercises, again derived from 
aviation simulation expertise.” Simulation creates a zero-risk environment 
that allows medical teams to practice high-risk, low frequency events 
without endangering patients.” This training can either be done in an 
artificial “laboratory” environment or “in situ’-training, which is conducted 
on actual patient care units involving actual health care team members and 
actual organization processes.” Simulation - if well debriefed - has many 
advan-tages, but if used as a stand-alone modus without the basis of CRM- 
training holds the risk of focusing too much on technical skills and single- 
task performance.” This will result in a limited impact on patient safety. The 
key to the success of team training tools in health care is the identification 
of the domain-specific team skills required for effectively managing 
routine and emergency scenarios effectively.” We suggest implementing 
two separate phases of simulation training: the first level of training mainly 
focuses on technical skills and then CRM-training (classroom) followed 
by second level simulation training that focuses on non-technical perfor- 
mance. 


4. CRM retention. Research in military aviation shows that retention of the 
CRM-subject matter and the effect on aircrew attitude degrades after 3 
years. Therefore CRM refresher training in the RNLAF is scheduled every 3 
years.” Whether the hospital setting calls for a similar refresher-schedule 
or regular well-debriefed simulation sessions are effective enough is still 
unclear. Chapter 6.1 of this thesis explores this issue. 


Conclusions 


Human Factors account for the majority of adverse events in aviation as 
well as in clinical medicine. The current safety paradigm is still based on 
ways to limit human variability in otherwise safe systems, promoting 
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stringent procedural guidelines. CRM focuses on improving interprofes- 


sional cooperation and team performance and thus patient safety. Even 


though evidence of CRM on medical errors and patient outcome is still 
scarce, the parallels between the critical processes in aviation and Intensive 


Care suggest that a well-adapted medical CRM training has potential for 


the ICU environment too. 


List of abbreviations 


AHRQ: 
CRM: 
CUSP: 
FAA: 
HIPAA: 
ICU: 
JAR: 
LOS: 
NASA: 
RNLAF: 
SA: 
SAQ: 


Agency for Healthcare Research and Quality (United States) 
Crew Resource Management 

Comprehensive Unit-based Safety Program 

Federal Aviation Authority (United States) 

Health Insurance Portability and Accountability Act (United States) 
Intensive Care Unit 

Joint Aviation Authority (Europe) 

Length of Hospital Stay 

North American Space Administration 

Royal Netherlands Air Force 

Situational Awareness 

Safety Attitudes Questionnaire 
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Abstract 


Background 

Human factors account for the majority of adverse events in both aviation 
and medicine. Human factors awareness training entitled “Crew Resource 
Management” (CRM) is associated with improved aviation safety. We 
determined whether implementation of CRM impacts outcome in critically 
ill patients. 


Methods 

We performed a prospective three-year cohort study in a 32-bed ICU, 
admitting 2500-3000 patients yearly. At the end of the baseline year all 
personnel received CRM training, followed by 1 year of implementation. 
The third year was defined as the clinical effect year. All 7271 patients 
admitted to the ICU in the study period were included. 

The primary outcome measure was ICU complication rate. Secondary out- 
come measures were ICU- and hospital length of stay, and standardized 
mortality ratio. 


Results 

Occurrence of serious complications was 67.1/1000 patients and 66.4/1000 
patients during the baseline and implementation year respectively, 
decreasing to 50.9/1000 patients in the post-implementation year (p=0.03). 
Adjusted odds ratios for occurrence of complications were 0.92 (95%CI 
0.71-1.19, p=0.52) and 0.66 (95%CI 0.51-0.87, p=0.003) in the implemen- 
tation and post-implementation year. The incidence of cardiac arrests 
was 9.2/1000 patients and 8.3/1000 patients during the baseline and 
implementation year, decreasing to 3.5/1000 patients (p=0.04) in the post- 
implementation year, while cardiopulmonary resuscitation success rate 
increased from 19% to 55% and 67% (p=0.02). Standardized mortality-ratio 
decreased from 0.72 (95%CI 0.63-0.81) in the baseline year to 0.60 (95%CI 
0.53-0.67) in the post-implementation year (p=0.04). 
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Conclusion 
Crew Resource Management-implementation is associated with a reduc- 
tion in serious complications and lower mortality in critically ill patients. 
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Introduction 


To err is human. As a result everything that a human being devises, uses, 
or does is prone to error and failure. Human factors refer to environmental, 
organizational and job factors, and to human and individual characteristics 
which influence professional behavior in a way that affects performance 
and safety.’ Human factors account for the majority of adverse events 
in aviation. Human factors awareness training entitled ‘Crew Resource 
Management’ (CRM) was introduced in 1979 for aircrew after a series of 
accidents in which human factors were found to be the root cause.’ 
Following the implementation of CRM, a decrease in events led to CRM 
becoming the present day aviation operational standard. Especially 
during time critical cockpit emergencies CRM is considered vital for air- 
crew effectiveness, although sound scientific proof of CRM effectiveness in 
aviation is lacking. 

In clinical medicine, human factor related errors can have a major impact 
on patient safety.*° This is especially so in departments where high risk, 
time critical procedures on vulnerable patients are performed in a multi- 
disciplinary team setting, human factors are likely to play an important 
role.’ Indeed, in critically ill patients the occurrence of complications is 
related to outcome.® 

The defining aspect of CRM is a system approach to safety culture. Rather 
than focusing on individual failure, CRM aims to identify system flaws and 
uses standardized communication tools to improve process effectiveness 
and safety’. The message of “good people are set up to fail in bad systems 
let's figure out how to keep everyone safe” is more easily accepted than “you 
have a problem that needs to be corrected”.'° This approach is fundamen- 
tally different from conventional quality and safety programs in medicine 
that focus on limiting variation in human behavior through regulations, or 
scenario-based team training.'''? Contrary to aviation, there is currently no 
international standard for medical CRM training. 
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While the parallels between the critical processes in aviation and medicine 
suggest that a well-adapted medical CRM training may have potential 
to improve patient safety, evidence of the effects of CRM on patient out- 
come in a clinical setting is limited. Closed format Intensive Care Units 
(ICUs) facilitate both CRM implementation and effect evaluation. The aim 
of this study was to assess the effects of CRM implementation on outcome 
in critically ill patients. 


Materials and Methods 


Department and training organization 

Radboud University Medical Center (Radboudumc) is a 953 bed, tertiary- 
care academic hospital with approximately 32,000 admissions annually. The 
32-bed ICU admits 2500-3000 patients yearly, of which approximately 1000 
are cardio-thoracic surgery patients. Radboudumc’s ICU workforce (in FTE) 
includes 16 consultants, 9 fellows, 22 residents, and a nursing staff of 173. 
The independent aerospace training organization that provided the CRM 
training consisted of five trainer/coaches (senior military and commercial 
airline pilots, psychologists and medical specialists of which some are dual- 
qualified). All were proficient in the area of operational human factors as 
well as CRM development and training. 


CRM intervention 

With the decision to implement CRM in the ICU, it was decided to evaluate 
its effects prospectively. Because of the nature of the intervention (team 
training), it was not feasible to perform a study using a control arm. For this 
reason the pre-during-post design was chosen. Therefore, a ‘baseline year’, 
‘implementation year’, and ‘post-implementation year’ were designated. In 
the two years preceding the study we had a stable baseline complication 
rate, but as we could not perform data quality checks retrospectively, we 
decided not to use these data, and only use the prospective data obtained 
during the baseline year prior to implementation of CRM. At the end of the 
baseline year, all ICU personnel (attending physicians, nurses and residents) 
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received a two-day (8 contact hours per day) CRM awareness training in 
multidisciplinary groups of a maximum of 15 participants within a three- 
month window. All training sessions were conducted by two trainers and 
were held at a training facility at some distance to the Radboudumc campus 
to minimize interference. The course included lectures on human factors 
and principles of CRM, and multiple interactive sessions using realistic data 
such as case studies and video footage from the Radboudumc ICU depart- 
ment. The training emphasized eight key areas: situational awareness'*"4 
and recognition of adverse situations, human errors and non-punitive 
response, communication and briefing and debriefing techniques, 
providing and receiving performance feedback, management of stress, 
workload and fatigue, creating and maintaining team structure and 
climate, leadership in a flat hierarchy environment, and risk management 
and decision-making (see'> or Supplemental File for more detailed infor- 
mation on training curriculum). Each training group created a shortlist of 
practical “action points” to be used in the following implementation year, 
during which the CRM principles were forged into custom-made and prac- 
tical clinical tools such as standardized briefings and debriefings, check- 
lists and noise (static) reduction methods. We considered the after-training 
follow-up to be crucial for the success of a CRM-intervention. 

The follow-up during the implementation year consisted of several 
measures: a “CRM Core Group” was formed, CRM became a standard item in 
staff meetings, non-facultative scenario based team training sessions were 
planned and CRM-training was provided to all new personnel. Moreover, 
regular process observation took place. 

The CRM Core Group consisted of three intensivists, 11 ICU nurses, and the 
ICU's patient safety officer. Under additional coaching from a CRM-instructor 
the Core Group coordinated translation of the CRM action points into clinical 
practice, and created professional ownership. This group interfaced regularly 
with the department’s medical and nursing staff and organized several 
activities such as a plenary kick-off meeting, an annual dedicated “week of 
CRM’, refresher lectures, and awarded the “CRM-performer of the year” The 
CRM Core Group posted regular information bulletins on the ICU intranet 
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page and developed several checklists for common ICU procedures, 
including central venous line placement, endotracheal intubation, tracheo- 
tomy and handover during patient admission and transfer. In addition, CRM 
became a standard agenda item during the two-weekly staff meetings and 
yearly individual evaluations. 

Furthermore, to secure the CRM lessons learned in daily practice, scenario- 
based team training was conducted during the implementation year. This 
team training was conducted in an in-hospital training location, using 
simulation manikins and CRM-trained clinical supervisors. As with the 
CRM-training, the scenario based team training was non-facultative, used a 
multidisciplinary setting and involved the whole staff. 

All new personnel (13 nurses and 2 Intensive Care fellows) also received the 
regular two-day CRM training. 

Finally, three times a year during a two-week period designated senior ICU 
nurses were given the responsibility to observe an ICU-unit whenever any 
invasive procedure took place. The resulting information was reported to 
the ICU's safety officer, who presented the collected info to the CRM core 
team. No data were provided to the staff during the data collection period. 


Data collection 

Patient clinical outcome data was collected from the Dutch National 
Intensive Care Evaluation (NICE) database.'® The NICE classification system 
is not based on Clavien-Dindo or the AMA-master classification system. The 
Dutch ICU complication registration was developed by a subcommittee of 
the Dutch Society for Intensive Care Medicine and started with a question- 
naire in which individual members were able to make suggestions for 
potential complications that should be included in the final registry. Out of 
68 potential complications a final list was constructed by the subcommittee 
based on the following prerequisites: 1) existing evidence in the literature 
that the complication relates to patient damage, 2) complication must be 
either severe or frequent, 3) the complication must be clearly defined and 
measurable, 4) the complication must be preventable by taking appropriate 
measures. The final list was approved by all Dutch Society for Intensive 
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Care Medicine members and included in the NICE registration.'©" All 
complications were registered daily by a consultant intensivist. To prevent 
incorrect (e.g. double) registration the data were subsequently checked by 
the departmental database manager and rechecked by the national NICE 
organization. Furthermore the NICE organization performs on-site data 
audits to monitor the quality of the data. Data was encrypted by the 
removal of all patient-identifying information. In the Netherlands, there 
is no need to obtain consent to use such registries with anonymous data. 
The NICE initiative is officially registered in accordance with the Dutch 
Personal Data Protection Act. The study was carried out in accordance with 
the applicable rules concerning the review of research ethics committees 
and informed consent. Data collection was standardized according to strict 
definitions and was subject to stringent quality checks. 

To determine the effect of CRM implementation on complication incidence, 
we used a predetermined set of 18 complications obtained from the NICE 
database.'*'” Data was collected during the baseline year preceding the 
three-month CRM training phase (August 2009 to end of July 2010), the 
implementation year following the training phase (November 2010 to end 
of October 2011), and the post-implementation year (November 2011 to 
end of October 2012). 

The primary outcome measure was overall ICU complication rate, of which 
the incidence of cardiac arrest and cardiopulmonary resuscitation (CPR) 
success rate had our special interest. Cardiac arrest incidence is considered 
a measure of quality of care’? and, as few clinical interventions rely on team- 
work to such an extent as CPR, outcome was deemed a relevant endpoint 
as well. Secondary outcome measures were ICU- and hospital length of stay, 
and standardized mortality ratio. Furthermore, as evidence from other critical 
industries’? and clinical settings?” suggests a positive relationship between 
safety climate and safety outcome we measured the ICU’s safety climate 
prior to and following the implementation year. Safety climate was 
determined by the Safety Attitudes Questionnaire (SAQ) using a Likert 
scale transformed to a 100-point scale.” The SAQ is a validated healthcare 
derivative of the Cockpit Management Attitudes Questionnaire.” 
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All CRM participants were invited to fill out a validated Dutch translation of 
the SAQ “(described in Chapter two of this thesis)” during the baseline year 
and at the end of the implementation year. During the implementation 
period process surveillance was conducted to monitor professional 
compliance: experienced observers scored 3-4 times per year during 
several weeks how many critical process opportunities suitable for CRM 
tools occurred and in what portion they were actually used. 


Statistical analysis 

Mann-Whitney U-tests were used to compare continuous data between 
two groups, Kruskall Wallis tests were used to compare continuous data 
between three or more groups, and chi-square tests were used to com- 
pare proportions. We used an uncorrected chi-square test to evaluate our 
null hypothesis. In addition, multivariate logistic regression analysis was 
performed using APACHE IV probability score as a covariate to correct for 
differences in disease severity. To correct for possible differences in baseline 
patient characteristics, standardized mortality ratios (SMRs), their 95% Cl, 
and differences between SMRs were calculated as described previously.”°”” 
A two-tailed p-value of less than 0.05 was considered statistically signifi- 
cant. Differences were not corrected for multiple testing because of the 
explorative nature of the study. With an estimated baseline complication 
rate of approximately 60-80/1000 patients, a power of 80%, and a two- 
sided alpha of 0.05, 2500 patients per cohort were required to detect an 
absolute change in complications of 20/1000 patients. Data was analyzed 
using SPSS Statistics 20 (IBM Corp., Armonk, NY, USA) and MedCalc 11.3.1.0 
(MedCalc software, Oostend, Belgium). 


Results 
Patient data 
The three cohorts consisted of 2295 (baseline year), 2423 (implementation 


year), and 2553 (post-implementation year) patients. During the three-year 
study period no relevant changes in staffing levels, device use, or protocols/ 
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procedures occurred, except for the ICU moving to another location in 
the hospital in December 2011. Patient characteristics are listed in Table 
1. A significant decrease in complication incidence rate was observed: 
from 67.1/1000 patients in the baseline year and 66.4/1000 patients in the 
implementation year to 50.9/1000 patients in the post-implementation 
year (Figure 1, Table 2). Using the APACHE IV probability score as a covariate, 
the adjusted odd ratios for the occurrence of one or more complications 
were 0.92 (95% CI 0.71-1.19, p=0.52) and 0.66 (95% CI 0.51-0.87, p=0.003) in 
the implementation and post-implementation year, respectively. 

Changes in the incidence per complication diagnosis are depicted in 
Table 2. The incidence of critical illness polyneuropathy/myopathy and 
cardiac arrests on the ICU decreased significantly. Using the APACHE IV 
probability score as a covariate, the adjusted odd ratios for critical illness 
polyneuropathy/myopathy were 0.52 [Cl 0.25-1.06, p=0.07] and 0.26 [CI 
0.11-0.63, p=0.002] in the implementation and post-implementation year, 
respectively. Adjusted odds ratios for cardiac arrests on the ICU were 0.87 
[Cl 0.47-1.62, p=0.66] and 0.33 [CI 0.15-0.73, p=0.006] in the implementation 
and post-implementation year, respectively. Apart from the decrease in 
incidence of cardiac arrests, the CPR success rate increased from 19% in the 
baseline year to 55% and 67% in the implementation and post-implemen- 
tation year (Figure 2). 

As expected, the occurrence of complications was associated with morta- 
lity (APACHE IV adjusted odds ratio of 1.97 [95% CI 1.44-2.70, p<0.0001)). 
Finally, the standardized mortality ratio was 0.72 (95% Cl 0.63-0.81) in the 
baseline year, 0.69 (95% CI 0.61-0.78) in the implementation year, and 0.60 
(95% CI 0.53-0.67) in the post-implementation year (baseline vs. post- 
implementation year: p=0.04). 

No clinically relevant effects on ICU- or hospital-length of stay were 
observed. 


Safety climate assessment 


Following CRM implementation, perceived safety climate significantly 
improved in 5 out of 6 Safety Attitudes Questionnaire-domains: teamwork 
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climate, safety climate, perceptions of management, working conditions, 
and job satisfaction (Table 3). Changes within disciplines (medical staff, 
nurses, other) are listed in Table 4 (Supplemental Digital Content). 


Process surveillance 

The surveillance from June and September 2012 showed that during 21 
observational days checklist use and briefing were performed in 70-90%. 
Debriefing performed between 55 and 71%. Structured handovers scored 
between 55 and 70%. 


Discussion 


This study indicates an association between CRM implementation in the 
ICU and a lower incidence of predefined complications in critically ill 
patients. Of special interest, cardiac arrests on the ICU occurred less 
frequently following implementation of CRM, and a higher CPR success rate 
was observed. 

In addition, we found that the occurrence of complications was associated 
with mortality and that implementation of CRM was associated with a 
reduced standardized mortality rate. These clinically relevant effects 
paralleled a positive impact on the perceived safety climate by the health 
care providers. 


Several factors may explain the positive effects of CRM implementation on 
clinical end points in the present study. In the first place, we introduced 
CRM from the perspective of correcting system flaws rather than indivi- 
dual shortcomings, an approach which has been shown to be effective 
in convincing professionals10. In addition, firm commitment and visible 
support from the department's leadership was present?! Secondly, 
every CRM-training was conducted by two trainers combining extensive 
operational experience in clinical medicine, military and commercial avi- 
ation, and cognitive psychology. As, in contrast to aviation, medical CRM 
has no accepted standard yet, their credible operational background was 
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instrumental in convincing ICU professionals of the potential gains of CRM 
and implementing a new professional and team identity in a department. 
Thirdly, a core group of ICU-professionals was formed to develop and 
integrate the new way of professional interaction within the ICU. As this 
group played a pivotal role in creating professional ownership, they 
received additional coaching from a CRM-instructor during the imple- 
mentation year and likely optimized training impact. Fourthly, to prevent 
dilution of human factors awareness the CRM intervention project included 
not only the initial training period, but also the subsequent training of new 
staff. Fifthly, some checklists were (re)designed. As checklists appear to be 
independent effective tools for improving patient safety?82, it is difficult 
to separate their impact on results from the CRM-effort. In spite of the fact 
that we cannot exclude a positive influence of a (re)designed checklist on 
our results we are convinced that the implementation of a checklist docu- 
ment has less effect on patient safety than the way the team works with 
the document (e.g. briefing and crosscheck techniques), especially because 
checklists were already in use at the department before the intervention. As 
we consider checklists a separate - if valuable - safety tool, the CRM-training 
focused on checklist use, not design. 

Finally, to secure the CRM lessons learned in daily practice, scenario-based 
team training was conducted during the implementation year.'*°° 


Several limitations of the present study need to be addressed. Most 
importantly, this study was a non-randomized single-centre study. The 
intensity and duration of the implementation process importantly limited 
the feasibility of other study designs. In spite of this limitation we believe 
that the decrease in complication rate and SMR can be related to the CRM- 
intervention. In the two years preceding the study baseline complication 
rate was stable, but as these registrations could not be validated retrospec- 
tively, this data was not used. Furthermore, during the whole study period 
there were no changes in interventions that are known to reduce morbidity 
or mortality in the ICU such as strict glucose regulation, early goal-directed 
therapy, use of corticosteroids, prone positioning and low tidal volume 


70 


ventilation. The fact that the ICU moved to another location in the hospital 
appears unlikely a confounder of our results, as no relevant changes in 
procedures, staffing levels, technical infrastructure or other major changes 
that could influence patient management occurred. Nevertheless, some 
differences in patient characteristics between the different study periods 
were observed. It could be argued that the risk of a complication is related 
to the severity of illness. Importantly, after correction for severity of illness, 
implementation of CRM was still associated with a reduction in relevant 
complications and mortality. The additional association between the 
occurrence of a complication and mortality supports the notion that CRM 
accounts for the bene-ficial effects on mortality observed. 

Additionally, the occurrence of critical illness polyneuropathy/myopathy 
is related to disease severity. As such, our finding of reduced incidence 
of critical illness polyneuropathy/myopathy is consistent with the notion 
that CRM is associated with improved care and reduced disease severity. 
Nevertheless, we emphasize that a direct link between the reduced 
incidence and the CRM implementation is not possible. 

In addition, a direct link between a specific action and patient outcome, 
e.g., using data logging equipment (comparable to aviation’s cockpit voice 
recording and flight data recording) was not determined. Even though 
video logging of ICU procedures would be technically possible, there still 
is considerable reluctance in the medical professional community due to 
legal and patient privacy issues.” 

Finally, because the Safety Attitude Questionnnaire’s response rate after 
training was relatively low (51%). Low response rates may increase the risk 
of a non-response bias. 


Perceived safety climate is positively related to safety outcomes both in 
hospital settings and other high-hazard fields.?'??? Previous studies on 
the impact of safety climate on safety outcome have focused on either the 
effect of team training on perceived safety climate***°, or assessed patient 
outcome. Implementation of CRM resulted in a culture change and a 
safer environment, illustrated by a decrease in malpractice expenses.” 
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To our knowledge, our concordant observation that complication rates and 
mortality decreased represents the first clear association between CRM 
training, clinical outcome, and perceived safety climate. To date, no 
prospective randomized trials evaluating the implementation of CRM are 
available. 

A large nationwide retrospective study, with a contemporaneous control 
group, in surgical patients reported a decrease in the overall mortality 
rate in (non-randomized) hospitals that participated in a training program 
focused on briefings and debriefings in the operating room (including the 
use of checklists), while no decrease in mortality rate was observed in the 
control group. In this study risk-adjusted mortality rates did not reach a 
statistically significant difference between the trained and non-trained 
institutes.” 


In conclusion, our data indicate an association between CRM implemen- 
tation and a reduction of complication rate and mortality in critically ill 
patients as well as an improved perceived safety climate. To our knowledge, 
this is the first study that links CRM to improved clinical outcome. In view 
of these results and absence of deleterious side effects for the patients, one 
might argue that, similar to aviation, widespread implementation of CRM 
in the ICU is justified, even without higher levels of evidence obtained from 
randomized clinical trials. 
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Figure 1 
Complication incidence per 1000 patients in the three cohorts. 
p-value calculated using Chi-square test. 
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Figure 2 

Cardiopulmonary resuscitation incidence (bars) and success percentage 
(dots/line) in the three cohorts. 

Cardiopulmonary resuscitation incidence significantly decreased (p=0.04, 
Chi-square test) while the success rate significantly increased (p=0.02, Chi- 
square test). 
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Tables 


Table 1: Patient characteristics 


Number of ICU patients (n) 
Age (yr) 

Female sex, no (%) 
Weight (kg) 

Height (cm) 


Type of admission 
Medical, no (%) 
Elective surgery, no (%) 
Emergency surgery, no (%) 


Chronic diagnoses 
Chron cardiovascular insuff, no (%) 
Resp insuff, no (%) 
Chron renal insuff, no (%) 
Chron dialysis, no (%) 
Metastaticsed malignancy, no (%) 
Hematological malignancy, no (%) 
Immunol insuff, no (%) 
AIDS, no (%) 


APACHE diagnosis group 
Cardiovasc, no (%) 
Gastro-intestinal, no (%) 
Genito-urinary, no (%) 
Hematology, no (%) 
Metabolic, no (%) 
Musculoskeletal/Skin, no (%) 
Neurological, no (%) 
Respiratory, no (%) 
Transplant, no (%) 
Trauma, no (%) 


Mechanical Ventilation in first 24 hrs, 
no (%) 


Vasoactive medication, no (%) 
APACHE-IV score 
APACHE-IV probability score 
SAPS 

SAPS probability score 

LODS 

LODS probability score 
MPMO 

MPM24 


Baseline year 
2295 

64 (53-73) 
818 (35.6) 

79 (70-89) 
172 (165-180) 


739 (32.2) 
1215 (52.9) 
341 (14.9) 


112 (4.9) 
65 (2.8) 
91 (4.0) 
28 (1.2) 
78 (3.4) 
70 (3.1) 
78 (3.4) 
2 (0.1) 


1263 (55.0) 
175 (7.6) 
55 (2.4) 

16 (0.7) 

19 (0.8) 

34 (1.5) 
301 (13.1) 
295 (12.9) 
12 (0.5) 
125 (5.4) 


1932 (84.2) 


992 (43.2) 
53 (40-69) 

0.05 (0.01-0.18) 
34 (27-44) 

0.15 (0.08-0.33) 
5 (4-7) 

0.21 (0.15-0.38) 
0.11 (0.07-0.22) 
0.24 (0.14-0.37) 


Implementation year Post-implementation year 


2423 

64 (53-73) 
865 (35.7) 

78 (69-88) 
173 (166-180) 


774 (31.9) 
1364 (56.3) 
285 (11.8) 


31 (1.3) 
45 (1.9) 
86 (3.5) 
33 (1.4) 
59 (2.4) 
59 (2.4) 
107 (4.4) 
5 (0.2) 


1336 (55.1) 
174 (7.2) 
59 (2.4) 

17 (0.7) 

27 (1.1) 

43 (1.8) 
305 (12.6) 
323 (13.3) 
13 (0.5) 
126 (5.2) 


2071 (85.5) 


1184 (48.9) 
63 (48-79) 

0.07 (0.03-0.22) 
36 (30-46) 

0.18 (0.11-0.37) 
6 (5-8) 

0.29 (0.21-0.48) 
0.09 (0.06-0.20) 
0.25 (0.15-0.37) 


2553 

65 (54-73) 
899 (35-.2) 
79 (70-90) 
174 (166-180) 


842 (33.0) 
1352 (53.0) 
359 (14.1) 


83 (3.3) 
68 (2.7) 
114 (4.5) 
50 (2.0) 
103 (4.0) 
54 (2.1) 
135 (5.3) 
0 (0) 


1289 (50.5) 
211 (8.3) 
64 (2.5) 

20 (0.8) 

30 (1.2) 

51 (2.0) 
334 (13.1) 
354 (13.9) 
21 (0.8) 
179 (7.0) 


2059 (80.7) 


1000 (39.2) 
66 (50-82) 

0.09 (0.04-0.25) 
36 (29-46) 

0.18 (0.37) 

6 (4-8) 

0.29 (0.15-0.48) 
0.10 (0.06-0.22) 
0.23 (0.13-0.38) 


p-value 


0.12 
0.93 
0.14 
0.049 


0.01 


<0.001 
0.07 
0.26 
0.08 
0.007 
0.11 
0.006 
0.06 


0.16 


<0.001 


<0.001 
<0.001 
<0.001 
<0.001 
<0.001 
<0.001 
<0.001 
<0.001 
0.10 


Data are presented as median (interquartile range) or number (percentage). p-value calculated by Kruskall Wallis tests or 


chi-square tests. 


Table 2: Complication incidence and outcome parameters 


Number of ICU patients 

Line sepsis 

Ventilator induced pneumonia 
Decubitus (grade III / IV) 
Unplanned extubation (self) 
Unplanned extubation (other) 
Acute myocardial infarction 
Cardiac arrest 

Pneumothorax (iatrogenous) 
CVA (stroke) 


Critical illness polyneuropathy / 
myopathy 


Difficult intubation” 
Loss of airway 


Early tracheostomy related 
hemorrhage 


Late tracheostomy related 
hemorrhage 


Loss of airway during trachea 
canula related procedure 


Anatomical complications with 
tracheostomy 


Vascular access problem 


Gastrointestinal bleeding 


Total no. of complications 


ICU-LOS (days) 
Hosp-LOS (days) 
ICU mortality, no (%) 
SMR 


Baseline year 


2295 
3 (1.3) 

1 (0.4) 
20 (8.7) 
40 (17.4) 
0 (0.0) 
13 (5.7) 
21 (9.2) 
10 (4.4) 
1 (0.4) 
21 (9.2) 


1 (0.4) 
3 (1.3) 
5 (2.2) 


2 (0.9) 


1 (0.4) 


1 (0.4) 


6 (2.6) 


5 (2.2) 


154 (67.1) 


1.0 (0.8-3.0) 


6.7 (3.7-15.2) 
187 (8.1) 


0.72 
(95% Cl 0.63-0.81) 


Implementation 
year 


2423 
7 (2.9) 

1 (0.4) 
17 (7.0) 
46 (19.0) 
1 (0.4) 
27 (11.1) 
20 (8.3) 
10 (4.1) 
2 (0.8) 
12 (5.0) 


4 (1.7) 
0 (0.0) 
4 (1.7) 


1 (0.4) 


0 (0.0) 


0 (0.0) 


5 (2.1) 


4 (1.7) 


161 (66.4) 


1.1 (0.8-3.0) 


6.1 (3.1-13.8) 
201 (8.3) 


0.69 
(95% Cl 0.61-0-78) 


Complication data are presented as: incidence (incidence/1000 patients). 
ICU-LOS, Hosp-LOS are presented as median (interquartile range). ICU mortality and Hosp mortality are presented as 


number (percentage). 


Post-implementation 
year 


2553 
3 (1.2) 
1 (0.4) 
11 (4.3) 
48 (18.8) 
1 (0.4) 
23 (9.0) 
9 (3.5) 
10 (3.9) 
4 (1.6) 
7 (2.7) 


3 (1.2) 
4 (1.6) 
1 (0.4) 


1 (0.4) 


0 (0.0) 


0 (0.0) 


3 (1.2) 


1 (0.4) 


130 (50.9) 


1.0 (0.8-2.8) 


6.7 (3.5-13.4) 
211 (8.3) 


0.60 
(95% Cl 0.53-0.67) 


p-value 


0.29 
1.00 
0.16 
0.91 
0.63 
0.12 
0.04 


0.97 
0.43 
0.01 


0.45 
0.17 
0.22 


0.73 


0.34 


0.34 


0.51 


0.22 


0.03 


0.008 


0.09 
0.98 


0.04* 


p-value calculated by Kruskall Wallis test or chi-square test, except for SMR data for which the difference between the 
baseline and post-implementation year was calculated as described previously. '®"° * Difficult intubation was defined as 
more than 3 intubation efforts or an intubation duration of >10 minutes. 


Table 3: SAQ scores of ICU professionals 


Stress recognition 
Teamwork climate 
Safety climate 


Perceptions of management 


Working conditions 


Job satisfaction 


Before CRM training 


(n=251) 
43 (0-90) 


69 (38-90) 
64 (29-97) 


58 (17-84) 
58 (17-86) 


69 (43-90) 


After CRM training 
(n=161) 


48 (1-84) 
76 (38-100) 
70 (29-94) 


64 (27-84) 
58 (11-90) 


74 (43-95) 


p-value 


0.12 
0.001 
<0.001 


<0.001 
0.009 


0.04 


Data are represented as median (range). p-value calculated by Mann-Whitney U-test. Response 
before and after CRM training was 72% and 51%, respectively. See Table S1 (Supplemental Digital 
Content) for SAQ scores per discipline. 


Table 4: SAQ scores of ICU professionals per discipline 


Medical staff 


Stress recognition 
Teamwork climate 

Safety climate 

Perceptions of management 
Working conditions 

Job satisfaction 


Nurses 


Stress recognition 
Teamwork climate 

Safety climate 

Perceptions of management 
Working conditions 

Job satisfaction 


Other 


Stress recognition 
Teamwork climate 

Safety climate 

Perceptions of management 
Working conditions 

Job satisfaction 


Before CRM training (n=25) 


Before CRM training (n=204) 


Before CRM training (n=22) 


Data are represented as median (range). 


51 (22-90) 
76 (62-90) 
69 (46-97) 
53 (17-74) 
58 (27-79) 
74 (53-90) 


43 (0-79) 
69 (38-90) 
64 (29-88) 
58 (17-84) 
58 (17-79) 
69 (43-90) 


40 (17-74) 
72 (55-83) 
64 (48-79) 
64 (43-84) 
56 (38-79) 
74 (53-90) 
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After CRM training (n=14) 


64 (38-97) 
79 (69-86) 
76 (55-82) 
64 (27-74) 
58 (48-79) 
74 (64-84) 


After CRM training (n=136) 


48 (1-84) 
72 (38-100) 
70 (29-94) 
64 (27-84) 
58 (11-90) 
74 (43-95) 


After CRM training (n=11) 


40 (17-79) 
76 (69-86) 
75 (61-82) 
66 (43-74) 
64 (53-79) 
76 (48-90) 


Supplemental file 
Crew Resource Management (CRM) training content 


Training setting: 

Two-day (16 contact hours) full time training using lectures, video-feedback 
and interactive exercises. Training was conducted in a facility outside of the 
hospital (to minimize interference with daily practice) in multidisciplinary 
groups of 15 trainees by two trainers. All levels of care providers (attending 
physicians, nurses, residents, etc.) were required to participate. 


Curriculum / training subjects: 

Core subject of the training is “Situational Awareness” (SA), defined as 
the ability to identify, process, and communicate the critical elements of 
information relevant to the team with regard to the mission. Team SA is 
considered to be the basis for effective decision making in safety-critical 
environments and a core competence for any professional team. Threats 
to SA are called “red flags” (eg, conflicting inputs, preoccupation, lack of 
communication, confusion, violations of policies or procedures, failure 
to meet targets or address discrepancies, fatigue, complacency, stress), 
which indicate an adverse situation and an imminent undesirable event are 
discussed. 


“Human Errors” explains how several error types may develop into inci- 
dents. The benefits of effective teamwork in identifying threats and deal 
with them before they become errors are shown, followed by techniques 
in preventing and counteracting the effects of errors. The use of a compre- 
hensive framework of standard operating procedures (SOP) and checklists 
is also discussed. Goal is to adopt a credible, non-punitive policy toward 
human error (not violations). 


“Communication” explains CRM team communication principles and 
describes how to execute them. Trainees learn the components of a 
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cross-check (monitor the situation, recognize red flags, communicate 
red flags precisely, and follow up with feedback) and practice effective 
assertive statements that command attention, convey concern, state the 
problem, and propose a solution. Trainees learn how to perform productive 
briefings before critical team activities. 


“Performance feedback” focuses on professional assertiveness and per- and 
post event feedback. 

Per-event feedback starts with careful consideration of timing and rele- 
vance of the message, followed by 3 levels of performance feedback. The 
first level requires the team member to formulate the message short, clear 
and non-blaming (“doctor/nurse, I’m not sure we did all the checks...”). The 
second level contains a key word that has a defined value (“doctor/nurse, 
I'm not comfortable with that decision...”). If this feedback is ignored, and 
the situation is considered unacceptable the last resort could be a request 
to “stop the procedure’, including eye contact and e.g. touching the arm of 
the person performing the procedure. 

Post event feedback (or debriefing) is defined as a structured conversation 
in which individuals describe the plan versus the outcome, their thoughts 
about what transpired, and what they plan to do differently should a similar 
situation recur. 


“Management of stress, workload and fatigue” addresses signs, symptoms, 
and causes of stress, fatigue, and circadian rhythms, and explains how 
disruptive these factors may be to human performance. Coping strategies 
are discussed. 


“Creating and maintaining team structure and climate” introduces trainees 
to behavioral group dynamics and how to recognize and counteract 
negative group tendencies. 

“Leadership” focuses on the impact of leader- and follower roles on team 


effectiveness. Hospital professionals learn to appreciate the difference 
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between technical capabilities and leadership skills. This element shows 
the gains of effective leadership and provides practical do's and don'ts. 
“Risk management and decision making” trains participants to balance the 
criticality of a decision against its time urgency by a systematic approach. 
This makes explicit the value of shared mental models in team decision- 
making. 


In summary, CRM training teaches team behavioral strategies with 
lectures and interactive sessions. Training enables participants to recognize 
how the skills enhance performance of individuals and teams by leveraging 
all knowledge and abilities present in the team. 


To be effective as a culture intervention the CRM training phase is part of a 
multi-stage effort including implementation support, as we have published 
previously.” 


Reference 

E1. Haerkens MH, Jenkins DH, van der Hoeven JG. Crew resource manage- 
ment in the ICU: the need for culture change. Ann Intensive Care 2012; 
2(1):39. 
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Abstract 


Objective 

Human Factors account for the majority of adverse events. Human Factors 
awareness training entitled Crew Resource Management (CRM) is associated 
with improved safety and reduced complications and mortality in critically 
ill patients. We determined the effects of CRM implementation in the 
Trauma Room of an Emergency Department (ED). 


Methods 

A prospective three-year cohort study was performed in a level 1 ED, admit- 
ting >12,000 patients annually (>1500 trauma-related). At the end of the 
baseline year, CRM training was performed, followed by an implementation 
year. The third year was defined as clinical effect year. The primary 
out-comes were safety climate, measured using the Safety Attitudes 
Questionnaire (SAQ) and ED length of stay. Secondary outcome measures 
were hospital length-of-stay, and 48-hour crude mortality of trauma 
patients. 


Results 

All 5070 trauma patients admitted to the ED during the study period were 
included. Following CRM implementation, safety climate significantly 
improved in 3 out of 6 SAQ-domains, both at the end of the implemen- 
tation and clinical effect years: teamwork climate, safety climate, and stress 
recognition. ED length-of-stay of these patients increased from 141 [102- 
192] in the baseline year to 161 [116-211] and 170 [128-223] minutes in the 
implementation and clinical effect years, respectively (p<0.05 vs. baseline). 
Hospital length-of-stay was prolonged by 1 day in the implementation and 
clinical effect years (p<0.05 vs. baseline), while mortality was unaltered. 


Conclusions 


Although CRM-implementation in the ED was associated with an improved 
safety climate, the time spent by trauma patients in the ED increased. 
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Introduction 


Background 

Human Factors refer to environmental, organizational and job factors, 
and to human and individual characteristics which influence professional 
behaviour in a way that affects performance, safety and human well-being. ' 
As Human Factors account for the majority of adverse events in aviation, all 
aircrew receive Human Factors awareness training entitled ‘Crew Resource 
Management’ (CRM).? During the implementation of aviation CRM since 
1979, a concordant decrease in events led to CRM becoming the present 
day operational standard.* * CRM is deemed vital for aircrew performance, 
especially during time-critical emergencies. CRM uses a system approach 
to safety culture that focuses on identifying system flaws rather than 
individual failure“ Sto achieve a “just culture”: a professional environment 
in which front-line operators and others are not punished for actions, 
omissions, or decisions taken by them which are commensurate with their 
experience and training, but where gross negligence, willful violations, and 
destructive acts are not tolerated.’ 

Human factors also appear to impact team performance and patient safety 
in clinical medicine.#'® Human error, if not mitigated, often leads to cycles 
of error and unintended patient states, which are dangerous and pre- 
cede the majority of harmful outcomes.'' Therefore, it appears plausible 
that especially departments where high risk, time-critical procedures are 
performed in a multidisciplinary team setting are likely to benefit from 
CRM." '? On the other hand, the effect of implementation of CRM on work- 
flow and time spent per patient are currently unknown. 


Importance 

Our previous study in critically ill patients showed that implementation of 
the CRM-team format is indeed associated with a positive impact on safety 
climate, a reduction in serious complications, and lower mortality.” 

Caring for trauma patients in the often hectic and chaotic setting of an 
Emergency Department (ED) is prone to error because of a number of 
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organizational and human factors. Time-critical decisions made by multiple 
health care professionals and consultants, frequent workflow interruptions, 
numerous hand-offs of care, reliance on verbal orders, and shiftwork- 
induced provider fatigue all contribute to a challenging environment for a 
healthcare team to perform effectively and safely.'*"° 


Goals of this investigation 

The primary aim of this study was to assess the effects of CRM-implemen- 
tation on safety climate and time spent in a Dutch level 1 Trauma Center's 
Emergency Department. Furthermore, we assessed the effects of CRM 
implementation on hospital length-of-stay and 48-hour crude mortality of 
trauma patients. 


Methods 


Study design and setting 

A unit-wide CRM intervention was implemented in the ED and analyses of 
its effects in the Trauma Room were planned prospectively. As the nature 
of the intervention (training phase followed by 1 year with a multitude of 
implementation efforts) limited the possibilities of performing a study 
using a control arm, a pre-during-post implementation design was chosen 
and a ‘baseline year, ‘implementation year, and ‘clinical effect year’ were 
designated. 

CRM was implemented in the ED of the Isala hospital, a 978 bed teaching 
hospital with approximately 47,000 admissions annually. The ED treats 
around 35,000 patients yearly, including approximately 1600 trauma 
patients. 


Selection of Participants 

Isala’s ED workforce (in FTE's) includes 7 emergency physicians, 16 residents, 
and a nursing staff of 53. All ED personnel, 5 (out of 7) attending trauma 
surgeons and 6 (out of 40) anesthesiologists participated in the CRM- 
training. The independent aerospace training organization that provided 
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the CRM-training consisted of 9 trainer/coaches (senior military and 
commercial airline pilots, psychologists, and medical specialists, some of 
which are dual-qualified). All were proficient in the area of operational 
human factors as well as CRM development and training. 


Interventions 

At the end of the baseline year, all ED personnel received two consecutive 
days (8 contact hours per day) of human factors awareness training in multi- 
disciplinary groups of 15 participants. All training sessions were conducted 
within a short (two-month) window to maximize impact and were held at 
a training facility at some distance to the hospital to minimize interference. 
The training curriculum was developed from aviation CRM-standards 
and adapted to clinical healthcare by the training organization, included 
lectures on human factors and CRM principles, multiple team exercises, and 
interactive sessions using realistic data such as case studies and video foot- 
age from the ED. 

Key areas of the CRM-training included situational awareness and 
recognition of adverse situations, human errors and non-punitive response, 
communication and briefing and debriefing techniques, providing and 
receiving performance feedback, management of stress, workload and 
fatigue, creating and maintaining team structure and climate, operational 
leadership, and risk management and decision-making (see'’ for more 
detailed information on the training curriculum). Each training resulted 
in a shortlist of practical “action points” to be realized in the following 
implementation year (for example standardized briefings, debriefings and 
handover during patient admission and transfer, checklists for common ED 
procedures such as endotracheal intubation, and noise/static reduction 
methods). 

As implementation of Human Factors principles into daily practice is 
challenging and relies heavily on clinical ownership and extensive follow- 
up', a multidisciplinary “CRM Core Group” was formed to coordinate and 
monitor the implementation efforts. This Core Group consisted of 2 emer- 
gency physicians, 3 ED nurses, 2 trauma surgeons, 1 anesthesiologist, 
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1 anesthesiology technician, and a CRM-instructor in a coaching capacity. 
Their mission was to create professional ownership by translating the CRM 
action points into useable tools for clinical practice and interface regularly 
with the department's medical, nursing and management staff. During the 
implementation year two modalities of additional training were provided. 

First, to establish the lessons learnt from CRM training within daily clinical 
practice, scenario-based team training was conducted. This simulation was 
conducted 2-3 times a week on site, using basic simulation manikins and 
CRM-trained clinical supervisors. As with the CRM-training, the scenario 
based team training used a multidisciplinary setting and involved the 
majority of the ED staff (participation of surgeons and anesthesiologists/ 
ICU consultants was limited). Second, to prevent “dilution” of Human 
Factors awareness, all new personnel (5 nurses, 5 residents and 2 emer- 
gency physicians during the period of study conduct) received the regular 
two-day CRM training. 


Methods and Measurements 

Patient clinical outcome data were collected from the Dutch National 
Trauma Registry (NTR) database.'? The NTR was developed in 2004 by the 
Dutch Trauma Society and every trauma center is responsible for supplying 
all trauma data to the NTR. To prevent incorrect (e.g. double) registration 
the data were subsequently checked by the ED’s database manager and 
rechecked by the national Trauma Registry organization. Data were 
encrypted by the removal of all patient-identifying information. The 
National Trauma Registry initiative is officially registered in accordance 
with the Dutch Personal Data Protection Act. The study was carried out in 
accordance with the applicable rules concerning the review of research 
ethics committees and informed consent. Data collection was standardized 
according to strict definitions and was subject to stringent quality checks. 


Data was collected during the baseline year preceding the two-month 


CRM training phase (November 2011 until the end of October 2012), the 
implementation year following the training phase (January 2013 until the 
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end of December 2013), and the clinical effect year January 2014 until the 
end of December 2014). 


Outcomes 

The primary outcome measure was the effect of CRM implementation on 
perceived safety climate and time spent in the ED. As evidence from non- 
clinical’? and clinical??? critical environments indicate a positive relationship 
between safety climate and safety outcome, we measured the ED's safety 
climate at the end of all three study periods. Safety climate was determined 
by the Safety Attitudes Questionnaire (SAQ). The SAQ is a validated health- 
care derivative of the Cockpit Management Attitudes Questionnaire, and 
responsive to interventions.”° 226 All CRM participants were invited to fill 
out an online Dutch translation of the SAQ (SAQ-NL, described in Chapter 
two of this thesis) at end of the baseline, implementation and clinical 
effect year. Secondary outcome measures were hospital length-of-stay, and 
mortality. 


Analysis 

Kruskall Wallis tests with Dunn-Bonferroni post-hoc tests were used to 
compare continuous data between the three groups. Chi-square tests were 
used to compare proportions. In addition, univariate multiple linear regres- 
sion analysis was performed using age and injury severity score (ISS) as 
covariates. Data were analyzed using SPSS Statistics 22 (IBM Corp., Armonk, 
NY, USA) and Graphpad Prism 5.0 (Graphpad Software, La Jolla, USA). 


Results 


Safety climate assessment 

Response rates for the three online questionnaires were 86%, 54% and 
83% after baseline, implementation and clinical effect year, respectively. 
Following CRM implementation, perceived safety climate significantly 
improved in 3 out of 6 Safety Attitudes Questionnaire-domains both at the 
end of the implementation and clinical effect year: teamwork climate, safety 
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climate and stress recognition (Figure 1). Job satisfaction and working 
conditions scores did not change. Perceptions of management scores 
showed a decreasing trend at the end of the implementation year, and 
were significantly lower at the end of the clinical effect year compared with 
baseline. 


Patient data 

The three cohorts consisted of 1772 (baseline year), 1614 (implementation 
year), and 1684 (clinical effect year) trauma patients, including severe 
trauma patients delivered by Helicopter Emergency Medical Services 
(HEMS) 36 (baseline year), 32 (implementation year), and 40 (clinical effect 
year). During the three-year study period, no relevant changes in staffing 
levels, device use, or protocols/procedures occurred, except for the ED 
moving to another location in the hospital in August 2013. 


Patient characteristics are listed in Table 1. Age was higher in the implemen- 
tation year compared with the baseline year. Although median levels were 
similar, the injury severity score (ISS) was significantly higher in the clinical 
effect year compared with baseline. 

Time spent in the ED was almost 20 minutes and almost 30 minutes longer 
during the implementation and clinical effect year, respectively. 
Furthermore, hospital length-of-stay was higher in the implementation 
and clinical effect years compared with baseline, while crude early morta- 
lity (within 48 hours after admission) was not different between the study 
periods. 

After adjusting for age and ISS, ED and hospital length of stay remained 
significantly higher in the implementation and post-implementation years 
compared with the baseline year (ED length of stay: B [95% CI]: 15'°?' and 
27 [22-32], respectively, both p<0.0001; hospital length of stay: B [95% Cl]: 
0.9 [0.4-1.4] and 1.0 [0.5-1.5], respectively, both p<0.0001). 
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Discussion 


This study demonstrates an association between CRM implementation in 
the ED and an improved perceived safety climate, but also with a prolonged 
length-of-stay of trauma patients at the ED. Several factors may explain 
the positive effects of CRM implementation on perceived safety climate in 
the present study. First, the introduction of CRM was not limited to a single 
training effort, but comprised a myriad of activities to create professional 
ownership.® 

The action points, formulated during all training sessions and based on 
CRM core items, were molded into the new way of professional interaction 
within the ED by a core group of ED-professionals under additional 
coaching from a CRM-instructor during the implementation year. Further- 
more, to secure the CRM lessons learned in daily practice, scenario-based 
team training was conducted during the implementation year'” 7, an 
annual “week of CRM” was introduced, and team performance became a 
standard item in staff meetings. Second, CRM-training and implementation 
support was conducted by instructors with credible operational experience 
in clinical medicine, aviation, and cognitive psychology. As, in contrast to 
aviation, medical CRM has no accepted standard yet, this helped convince 
clinical professionals of the potential gains of CRM and allow this new 
professional and team identity into the ED. Third, to prevent dilution of 
Human Factors awareness CRM training was provided not only during the 
initial training period, but also to new staff during the later study periods. 
Finally, as the ED was the first department in the hospital that aimed to 
adopt Human Factors principles, firm commitment and endorsement from 
the department's manager and lead ED physician was instrumental throug- 
hout the intervention.® '” 


Safety climate is positively related to safety outcomes in high-hazard 
working environments, including the high-risk clinical arenas.*" > 3" In 
previous clinical studies on the impact of improved safety climate on out- 
come, team training was associated with improved safety climate? * 33, 
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patient outcome™, or both." Importantly, the number of studies that 
determined both safety climate and clinical outcome is limited to only one. 
Clinical CRM implementation in has also been associated with a reduction 
of malpractice expenses, reported to be as high as 62% in an obstetric 
environment.” One study in the operating room of an academic center 
even showed a dramatic cost reduction from $ 793,000 to zero.” To date, no 
prospective randomized trials evaluating the impact of CRM in the trauma 
room have been published. Interestingly, in contrast with improving 
overall SAQ-scores, the perceptions of management domain showed a 
decreasing trend. This may be indicative of the challenge that, as the 
clinically-led CRM-initiative gained momentum and awareness of Human 
Factors-related issues increased, managerial limitations to implementation 
of CRM action points emerged. CRM training may result in more safety 
awareness and a more critical appraisal of health care providers towards 
their management. As Paine et al described”, the clinician's concern about 
managerial work may present an obstacle to the implementation of human 
factors principles in healthcare. 


Evidence the impact of CRM on mortality is still scarce, and results vary. 3” 
In the present study, no effect on crude mortality was found. This is not 
surprising in view of the statistical power of the study and may be explained 
by the low mortality rate in our patient group with a mild overall ISS of 9. As 
previous studies of patients with mild trauma injury have shown, impacting 
mortality would require much larger cohorts than the present study.” Our 
study was underpowered to demonstrate an effect of CRM implementa- 
tion on outcome in the subgroup with a higher ISS. Hospital length-of-stay 
was significantly increased in the implementation and clinical effect years. 
We do not have an explanation for this observation and no factors linking 
our intervention to increased hospital length of stay were identified. ED 
length-of-stay was higher in the implementation and clinical effect years 
compared with baseline. In our view it is plausible that this is the result of 
CRM-implementation, as more stringent use of procedures and checklists 
may take more time. So, while thorough use of procedures and checklists 
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has benefits”, it may be more time-consuming. Although the current 
study provides no causal evidence linking CRM implementation to 
improved outcome, we support more widespread implementation of 
Human Factors awareness training in the high-risk ED environment. An 
interesting parallel, in aviation CRM became the present day operational 
standard due to the perceived impact on safety climate and aircrew 
effectiveness, especially during time critical cockpit emergencies, even 
though scientific evidence is still lacking. As the initial care for trauma 
patients in most level 1 ED’s adheres to the same treatment principles, our 
findings are likely to be generalisable to other international EM settings 
with comparable levels of patient activity. The current study has several 
limitations, two of which concern our study design. Most importantly, this 
is a non-randomized single-centre study. The intensity and duration of 
the training as well as the implementation process importantly limit the 
feasibility of using other study designs. Additionally, the fact that the safety 
climate assessment relied on anonymous completion of questionnaires did 
not allow for a individually paired analysis of the results. In spite of both 
these design limitations, we believe that it is plausible that the improved 
safety climate is related to the CRM intervention. Furthermore, the Safety 
Attitude Questionnaire’s response rate after implementation was relatively 
low. While low response rates may increase the risk of a non-response bias, 
results from “low” response-rate surveys may still accurately represent 
attitudes of the population”? This appears to be the case in the current 
study as well, as the last SAQ-measurement (following the clinical effect 
year), which resulted in an adequate response rate, showed a continuation 
of the previous trend. 

An important success factor for culture interventions is a stable, intact 
team.” As the ER is not a closed-format department and must respond to 
emergencies that require a broad spectrum of specialist expertise, main- 
taining a stable team is not possible, as ad hoc participation of physicians 
other than those of the trauma team is frequently required. In our setting, 
it proved challenging to make every professional with responsibilities in 
the ER beyond the ER- and trauma staff (e.g. anesthesiologists and trauma 
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surgeons) participate in CRM-training. This may have limited the impact of 
our intervention. Finally, the ED moving to another location in the hospital 
during the CRM-implementation phase may have confounded our results, 
even though no relevant changes in procedures, staffing levels, technical 
infrastructure, or other major changes that could influence patient 
management took place. The fact that the SAQ-domain of ‘working 
conditions’ showed no significant change over time supports that this was 
not a major factor. 

In conclusion, our study demonstrates that CRM implementation at the ED 
is associated with an improved perceived safety climate at the expense of a 
longer ED length-of-stay for trauma patients. 
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Tables 


Table 1 


Patient characteristics and clinical outcome 


Number of 
patients 


Age (years) 


Female sex 


ISS 


ED LOS 
(minutes) 


Hospital LOS 
(days) 


baseline year 


1772 


55 (23-76) 


822 (46.4) 


9 (4-9) 


141 (102-192) 


2 (1-7) 


Mortality within 21 (1.2) 


48 hours 


implementation year 


1614 


59 (28-79)* 


778 (48.2) 


9 (4-9) 


161 (116-211)* 


3 (2-8)* 


21 (1.3) 


clinical effect year 


1684 


57 (25-79) 


787 (46.7) 


9 (4-9)* 


170 (128-223)* 


3 (2-9)* 


33 (2.0) 


Data are presented as median (interquartile range) or number (percentage). 
p-values were calculated using Kruskall Wallis tests or chi-square tests. 

* indicates p<0.05 compared with baseline year (calculated using Dunn-Bonferroni post-hoc tests). 
ISS: Injury severity score; ED: emergency department; LOS: length-of-stay. 
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p-value 


0.007 


0.54 


0.006 


<0.0001 


<0.0001 


0.13 


Figures 
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Figure 1 


SAQ scores in the baseline, implementation, and clinical effect years 
Data are represented as median + interquartile range. * indicates p<0.05 


compared with baseline year (calculated using Kruskall Wallis tests follo- 
wed by Dunn-Bonferroni post-hoc tests). 
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Abstract 


Background 

Human factors account for the majority of adverse events in both 
aviation and medicine. We determined whether implementation of a 
Human Factors awareness training entitled “Crew Resource Management 
(CRM)” is associated with improved safety climate. 


Methods 

An observational 6-year study in 19 critical clinical departments in three 
university and seven teaching hospitals in the Netherlands that participa- 
ted in a CRM intervention. 

To assess changes in safety climate, all personnel were invited to fill out the 
Safety Attitudes Questionnaire (SAQ-NL) before the CRM-training (baseline 
measure, n=1412), one year after receiving CRM training (post-implemen- 
tation measure, n=558), and two years after training (follow-up measure, 
n= 270). 

Furthermore, factors for successful CRM implementation were analyzed 
and we evaluated points for improvement healthcare professionals wanted 
to implement after receiving initial CRM training, and possible existing 
barriers to transfer of the CRM training to the work floor. 


Results 

The overall response rates averaged 68.7%, 41.0% and 77.7%, respectively. 
There was a positive effect of CRM training on all six SAQ-NL domains over 
time (p<0.001), with a medium effect size. 

The success factors ‘strong clinical leadership’ and ‘participation of 90-100% 
of managers in the training’ were associated with a positive effect on the 
Perceptions of Management domain. The availability of a complication 
registration was associated with improved Working Conditions domain 
scores. Furthermore, integration of simulation training into CRM imple- 
mentation was related to improvements in Safety Climate, Perceptions of 
Management, and Working Conditions domains. 
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The results from points for improvement and process observations 
revealed that clinical professionals perceive the need for mandatory 
(de)briefings and more structured patient handovers, as well as improved 
individual leadership/followership and performance feedback skills. 


Conclusion 

Our data indicate an association between CRM implementation and an 
improved safety climate, and we identified factors that are important for 
successful CRM implementation. This could facilitate further optimization 
of healthcare CRM implementation, and increase awareness of possible 
barriers. 
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Introduction 


To err is human 

Medical error is estimated to be the third leading cause of death in the 
United States with over 250,000 patients having died as a result of 
preventable adverse events in 2013. The actual numbers may even be 
higher because underreporting is likely, as standardized root cause analysis, 
reporting of inpatient deaths and national data sharing is limited.' Beyond 
the psychological impact on the bereaved and the professionals involved, 
the financial costs due to these errors are estimated to range between $17 
and $29 billion per year in the United States.’ 


Human Factors awareness 

Although some medical errors can be attributed to technical malfunctions 
and failing equipment, many may find their roots in Human Factors (HF). 
It may therefore be reasonable for the healthcare community to improve 
HF awareness. An example of a mature HF awareness training curriculum, 
entitled Crew Resource Management (CRM), originated in the aviation 
domain. CRM was first introduced in a NASA workshop following research 
on the contribution of HF in aviation accidents. From there on, CRM evolved 
‘on the fly’ Currently, CRM revolves around the premise that human error is 
unavoidable and that error is a source of valuable information. The intent is 
to be aware of human performance limits, to avoid error or trap errors in an 
early stage, and to mitigate the consequences of errors that do occur. CRM 
utilizes communication tools to improve process effectiveness and safety. 
Aviation has very clear guidelines on CRM training content and frequency 
of recurrency training. Compliance with these guidelines is a license and 
currency requirement. © 


Safety Culture 

Safety culture is defined as “the product of individual and group values, 
attitudes, perceptions, competencies, and patterns of behavior that deter- 
mine the commitment to, and the style and proficiency of, an organization's 
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health and safety management”? CRM aims to achieve a “just” rather 
than a “no-blame” safety culture. Although the term no-blame® is often 
associated with CRM, it implicates that individuals should never be blamed. 
Just culture is based on the assumption that no complex system can be 
entirely flawless, but gross negligence or substandard performance should 
be reprimanded and individual professionals held accountable for their 
actions." 


Safety Climate 

As safety culture is only measurable by careful, long-term observations, it 
may be more appropriate to use the term safety climate.'' Safety climate 
refers to current employee beliefs, perceptions, and attitudes concerning 
safety and may be measured through questionnaires"', of which the Safety 
Attitudes Questionnaire (SAQ) is a validated and widely used example for 
healthcare.'” "> 


CRM in healthcare 

Each healthcare professional carries the responsibility to operate compe- 
tently, safely and in the best interest of the patient. However, as safe care 
relies on the interdisciplinary performance of a care team as much as on 
individual expertise, improving safety climate may result in a sound basis 
for better patient outcomes." Since the 1990's, CRM has been implemented 
in healthcare.” ">. 16 As a standard for medical CRM is still lacking, CRM- 
training comes in various guises. Even though this makes comparing 
studies on CRM in healthcare challenging, most publications suggest that 
CRM may be beneficial for safety climate in medical care. Positive effects 
were found on clinical error rate”, team climate, safety climate, stress 
recognition'®, technical and non-technical skills, efficiency’, team behavior 
and communication? and serious complications and mortality”! While 
most participants evaluate the CRM-training positively, organizational 
barriers to implementation”, compliance issues? and challenges in 
transfer of learned behaviours to the job™ have been identified. 
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Aims of this study 

The aim of the current study was to evaluate the effect of a CRM interven- 
tion on safety climate in multiple departments in several Dutch hospitals. 
Furthermore, we performed an explorative analysis to identify factors 
associated with successful implementation. The development of safety 
climate was assessed over a two-year period after training to provide 
reference for recurrency training schedules. 

Finally, we explored which safety measures healthcare professionals want 
to implement after receiving initial CRM training and possible existing 
barriers to transfer of training to the work floor. This could provide future 
CRM trainers, trainees, and policy makers with context specific information 
on how to further optimize implementation of healthcare CRM, and 
increase awareness of possible barriers. 


Methods 


Participants 

Participants were recruited at 19 critical clinical departments in three 
university and seven teaching hospitals in the Netherlands that partici- 
pated in a CRM intervention. Questionnaire data was collected pre-training 
(baseline survey), one year post-training (post-implementation survey), 
and two years post training (follow-up survey) in the period from October 
2010 to September 2016. Although only baseline and post-implementation 
questionnaires were part of the intervention, departments were asked 
if they were willing to participate in an additional follow-up survey one 
year later. Participants’ needs for change were investigated by analyzing 
lists that comprised improvement points suggested by trainees during 
CRM training. Additionally, five departments were invited to participate in 
process observations in January and February 2016. 


CRM intervention 


The CRM-intervention consisted of a preparation phase, a training phase 
and a subsequent implementation phase. 
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During the preparation phase, process observations were conducted by 
a minimum of three CRM trainers on separate occasions to evaluate the 
department based on CRM items as provided in the training. This also 
allowed trainers to get a ‘feel’ for the work environment and to collect 
examples of work processes. 


The training phase consisted of mandatory two consecutive 8-hour CRM 
training days and was given on an off-site location to minimize inter- 
ference. Each training was conducted by two trainers in multidisciplinary 
groups representative of the professional composition in day-to-day work 
(a typical group consisted of nurses, residents, and attending physicians) 
and was limited to 15 participants. The independent aerospace training 
organization that provided the CRM training consisted of six trainer/coaches 
(senior military and commercial airline pilots, psychologists and clinical 
consultants, or a combination thereof). All were proficient in the area of 
operational HF as well as CRM development and training. The core sub- 
jects were ‘situational awareness, ‘human error and non-punitive response’, 
‘communication and performance feedback, ‘stress and workload 
management; ‘group processes, and ‘leadership. The training included 
lectures, video footage, interactive exercises and adverse event case 
discussions. Please see supplemental file 1 for more detailed info on the 
training curriculum. Every trained group comprised a list of possible points 
for improvement on the work floor, based on training content and group 
discussions. The compilation of the points for improvement formed the 
basis for the subsequent implementation phase. 

The implementation phase started after all training sessions were 
completed. A “CRM core group” was formed from volunteer trainees. The 
core group received the compilation of points for improvement from all 
training sessions and was responsible for the translation of these points into 
useable clinical tools, thus initiating a change process based on the 
acquired CRM-knowledge and creating professional ownership. This group 
interfaced regularly with the department’s management team and received 
coaching from a CRM trainer during one year. 


Safety Climate measurement 

The Safety Attitudes Questionnaire (SAQ) is a reliable instrument to measure 
healthcare professionals’ attitudes, identifying six safety climate related 
domains: Teamwork Climate, Safety Climate, Stress Recognition, Job 
Satisfaction, Working Conditions, and Perceptions of Management (see 
supplemental file 1 for SAQ domain definitions).'? Safety climate measure- 
ments in this study were performed using a validated Dutch translation of 
the SAQ (SAQ-NL)."? 


Success factors 

A list of seven empiric success factors for implementation, obtained from 
our past CRM / HF integration projects (25) was compiled before analysis. 
These factors were: hospital administrator involved, strong clinical 
leadership supportive of CRM, CRM core group still active after one year, 
complication registration available, simulation training integrated as 
follow-up, 90-100% of professionals participated, and 90-100% of 
operational managers participated. Each department scored a 0 (absent) or 
a 1 (present) on each of these success factors. 


Points for improvement 

Every trained group composed alist containing possible points for improve- 
ment on the work floor, based on training content and group discussions. 
A total of 94 of these lists were analyzed. Items were categorized in the 
respective CRM training subject and categorized to provide an overview 
within the main category. 


Process observations 

The additional observations in the current study were performed by 
a trained observer in one university and three teaching hospitals. The 
observed departments consisted of three cardiac catheterization labs 
(CCL) and one operating room (OR). Observations lasted between five and 
seven and a half hours. Furthermore, when the opportunity arose, 
employees were asked the question “After receiving the CRM training, what 
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did and did not change in your job?” The main goal of the observations was 
to identify barriers to implementation of points for improvement. 


Data collection 

Approximately one month prior to the first training, participants were 
invited to fill out the SAQ-NL (baseline survey) and provide additional 
demographic data. When an entire department had received the CRM 
training, the CRM core group was formed and coached during one year. 
After this year, participants again received an invitation to fill out the SAQ- 
NL (post-implementation survey) and provide demographic data. A final 
invitation was sent two years after receiving the CRM training to depart- 
ments that volunteered in participating in the third measurement (follow- 
up survey). The points for improvement from all included CRM interventions 
were comprised from the lists each CRM core group received. For the 
process observations, five departments that participated in the CRM inter- 
vention were contacted and asked if they were willing to participate in an 
additional observation. They were informed on the aim of the current study 
and that the observation would be similar to the observation prior to the 
CRM training. 


Statistical Analysis 

Multivariate analysis of variance (MANOVA) was utilized to test the hypo- 
thesis that CRM training had a positive effect on safety climate in general, 
with the measuring moment (baseline, post-implementation, and follow- 
up) as independent variable and mean score on the six SAQ domains as 
dependent variables. Separate univariate ANOVA's were performed to 
analyze the effect of measuring moment on the mean scores of the 
separate SAQ domains. To test how safety climate changed during the 
first and the second year, a repeated contrast was utilized. This contrast 
compares the post-implementation to the baseline measure and the 
follow-up to the post-implementation. Furthermore, 
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we performed an explorative analysis using separate univariate ANOVA's to 
analyze whether the presence of each of the seven success factors mode- 
rated the relationship between measuring moment and the mean scores 
of the separate SAQ domains. Data were analyzed with SPSS 22 (IBM corp., 
2013). A p-level of p < 0.05 was considered to indicate statistical significan- 
ce. Furthermore, the following cut-offs were utilized for effect sizes: small 
effect, Cohen d = 0.2, n = 0.01; medium effect, Cohen d = 0.5, n, = 0.06; 
large effect, Cohen d = 0.8, n,* = 0.14. 


Results 


Participants 

The sample consisted of 19 departments in the baseline, 12 in the post- 
implementation, and two in the follow-up measure. In total, 2457, 1750 
and 405 professionals were invited to participate in the baseline, post- 
implementation and follow-up measure, respectively. The overall response 
rates averaged 68.7%, 41.0% and 77.7%, respectively. After excluding 
respondents that answered less than 28 of the 30 questions for further 
analysis, data were obtained from 1412, 558 and 270 participants in the 
baseline, post-implementation, and follow-up measure, respectively. An 
overview of participant gender and age, participants per academic and 
teaching hospital, hospital departments, and professional occupation is 
provided in Table 1. 


Safety climate assessment 

The main hypothesis was that CRM would have a positive effect on safety 
climate measured by SAQ-NL. The MANOVA test indicated that there was 
a significant effect of measuring moment (baseline, post-implementation, 
follow-up) across the six SAQ-NL domains, V = 0.12, F(12, 4466) = 22.6, p 
< 0.001, n = 0.06. This represented a significant overall improvement in 
safety climate with a medium effect size. 
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Separate univariate ANOVA's revealed that all six individual SAQ-NL domains 
improved after CRM implementation: Teamwork Climate, F(2, 2237) = 53.60, 
p < 0.001, n,? = 0.05, Safety Climate, F(2, 2237) = 53.52, p < 0.001, n? = 0.05, 
Job Satisfaction, F(2, 2237) = 41.48, p < 0.001, No = 0.04, Stress Recognition, 
F(2, 2237) = 16.05, p < 0.001, n, = 0.01, Perceptions of Management, F(2, 
2237) = 46.51, p < 0.001, n = 0.04, and Working Conditions, F(2, 2237) = 
36.54, p < 0.001, n = 0.03. Effects on the individual SAQ-NL domains are 
depicted in Figure 1. Furthermore, the mean scores on the SAQ-NL domains 
for every measuring moment and the effects sizes for the comparison of 
baseline vs. post-implementation and post-implementation vs. two year 
follow-up can be found in Table 2. These results indicate that CRM generally 
has a positive effect on safety climate both in the implementation and 
the follow-up year. There were no improvements in Job Satisfaction and 
Working Conditions in the first year, but Teamwork Climate, Safety Climate, 
Stress Recognition, and Perceptions of Management all significantly 
increased. Interestingly, there was a strong improvement on most domains 
in the follow-up year, except for Stress Recognition, which showed a mode- 
rate but statistically significant increase, and Perceptions of Management, 
which showed a trend towards improvement. 


Moderating effects of success factors on safety climate 

An explorative analysis was performed to evaluate the moderating effects 
of success factors on individual SAQ-NL domains. For this analysis, the 
dataset was limited to departments that provided both a baseline and a 
post-implementation measurement (the follow-up measure was not 
utilized). The abovementioned analyses performed on this data subset 
yielded virtually identical results as those found in the entire dataset (data 
not shown). To identify the moderating effects of the presence of the 
success factors, we evaluated the interaction term success factor* 
measuring moment for each of the seven success factors on each SAQ-NL 
domain. This analysis revealed that hospital administrator involvement, an 
active core group after 1 year, and participation of 90-100% of professionals 
were not related to improved SAQ-NL scores. Strong clinical leadership was 
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associated with a positive effect on the Perceptions of Management 
domain (F(1, 1667) = 18.77, p < 0.001, n,* = 0.011), and the availability 
of a complication registration was associated with improved Working 
Conditions domain scores (F(1, 1667) = 5.71, p = 0.02, n = 0.003). 
Integration of simulation training into CRM implementation was related to 
improvements in Safety Climate (F(1, 1667) = 5.19, p = 0.02, n= 0.003), 
Perceptions of Management (F(1, 1667) = 3.92, p = 0.048, n = 0.002) and 
Working Conditions domains (F(1, 1667) = 9.85, p = 0.002, n = 0.006). 
Finally, participation of 90-100% of managers in the training was associated 
with improved Perception of Management scores (F(1, 1667) = 18.24, p < 
0.001, n, = 0.011). These interaction effects are visualized in Figure 2. 


Points for improvement 
An overview of most cited points for improvement is provided in Table 3. 

Situational Awareness. To enhance Situational Awareness, most 
groups (83%) suggested implementation of mandatory briefing before and 
debriefing after a medical procedure to introduce participants, to distribute 
tasks and roles, to enhance the chances of particular information being 
shared, to create an opportunity to ask questions, to plan for contingencies, 
and to evaluate whether the procedure went as planned. 31% Suggested 
that structuring handover of patients should improve to prevent losing 
critical information when patients transfer from one department to 
another. 

Communication. The majority of the groups felt the need to lower 
the barrier of speaking up when patient safety was compromised (64%), or 
to address wanted and unwanted behavior in a neutral and de-escalating 
manner (17%). Lowering the communication threshold by addressing a 
colleague by name should be implemented according to 35%. Redu- 
cing noise, for instance by limiting the amount of people attending to a 
procedure or by banning phones and pagers from procedure rooms, was 
suggested by 34% of the groups. 27% Found improvement of proper 
closed-loop communication important to ensure that a receiver interpreted 
the message as it was intended by a sender, and to ensure that challenges 


116 


(e.g. patient date of birth, medication) are communicated as open- 
ended questions (e.g. “what is your date of birth’, “what medication am | 
administering’). 

Stress. 23% of the groups found it important to improve the 
opportunity to discuss traumatic events, such as witnessing death of an 
infant patient, either with mental health professionals or with colleagues. 
Furthermore, 16% would find it helpful to be able to share and be aware of 
personal situations that might influence or limit a colleague’s performance. 

Group Processes. The main topics concerning Group Processes 
that were transferred to points for improvement related to conflict and 
Standard Operating Procedures (SOP). 17% Of the groups suggested 
reduction of conflict by reducing gossip and promoting to talk with, rather 
than about each other. Implementing mandatory debriefing was seen 
as a tool to address differences of opinion or work methods in order to 
avoid escalation of conflict. 13% Found it useful to formalize assumed 
working agreements into SOP to reduce briefing times (only non-standard 
proceedings and particularities are briefed), to increase predictability of 
behavior, and as a tool for addressing others’ wanted or unwanted behavior. 

Leadership. The relevance to improve operational Leadership 
behavior was formulated by 37% of the groups. Especially in multi- 
disciplinary settings such as an ER, it is not always clear who is in charge. 
This can result in situations where either nobody feels in charge or multiple 
team members assuming they are in charge. 

Human Error. The main point for improvement (37% of the groups) 
regarding Human Error was to create a work environment in which people 
feel safe and have time to share and discuss mistakes and near misses with 
colleagues so that others can learn from them and future adverse events 
may be avoided. 


Process observations 


Several barriers to implementation of CRM on the work floor were observed 
on individual, team and organizational levels: 
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First, debriefings were often skipped due to time pressure to get to the next 
patient. In one case, a nurse stated that they no longer had time to debrief 
after a nurse position got cut from the team in a reorganization. Another 
reason for not debriefing was that the team broke up before ending the 
procedure, for instance when a surgical team left when their part of the 
procedure was done and the procedure ended with only the anesthesio- 
logy team present. 

Second, a department had merged with another hospital and now half 
of the staff had never received CRM training. According to one nurse this 
resulted in differences in who is open for feedback and criticism, and who 
is not. 

Third, two CRM core groups went dormant. One after increased work pres- 
sure due to a reorganization and there was no time to get together on a 
regular basis. The other shortly after coaching by the CRM trainer ended. 
Fourth, following the CRM intervention, there was usually no follow-up 
training for newly hired staff, either due to a lack of planning or due to 
budget cuts. 

Fifth, CRM is, by its very nature, work place oriented, therefore obtaining 
and retaining commitment from the department’s managers has proved 
challenging in several hospitals. 


Discussion 


The main finding of this study is that SAQ-NL measurements showed a 
significant improvement of all 6 domains in the two years following CRM 
implementation. Furthermore, our explorative analyses of empiric success 
factors suggest that especially integration of simulation training is an 
important determinant of successful CRM implementation. Finally, the 
large majority of trained professionals indicated that mandatory (de)brie- 
fings should become an intrinsic part of critical procedures, as should 
improved techniques for performance feedback. 
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The safety climate measurements revealed that 4 out of 6 SAQ-NL domains 
had improved in the post-implementation measurement, increasing 
to all SAQ-NL domains in the follow-up measurement. A possible 
explanation for the further increase of safety climate in the follow up year 
is that it takes time for a core group to think of, and implement, practical 
solutions to the ‘points for improvement”. During the first implementation 
year professionals see little change on the work floor initially, which 
may explain the absence of effects on the Job Satisfaction and Working 
conditions domains in the post-implementation measurement. The second 
year professionals may notice more changes taking effect and developing 
into an improved standard of working. Our finding that safety climate is still 
improving after two years may indicate that a three-year CRM-“refresher 
training cycle” for all personnel may suffice. Furthermore, the CRM 
intervention in the current study instigates bottom-up change by having 
trainees formulate the key issues they wish to address and forming a CRM 
core group that aims at dealing with these issues. This bottom-up process 
might heighten feelings of job autonomy, which increases job satisfaction 
according to the job characteristic model.”* 7” As perceived organizational 
support is important for employee safety involvement? and the oppor- 
tunity for CRM training may convey the management's concern for a safe 
work environment, this may explain the positive impact on Perceptions 
of Management. The increase in Perceptions of Management decreased 
in the follow-up year. This may highlight the importance of continuing 
support of management, especially after the initial intervention ended.” 


Evaluation of the moderating effects of empiric success factors derived 
from past CRM / HF integration projects (25) on SAQ-NL domains resulted 
in the finding that integration of simulation training into CRM implemen- 
tation was related to improvements in Safety Climate, Perceptions of 
Management as well as Working Conditions domains. Additionally, our 
finding that hospital administrator involvement is not associated with 
improved safety climate outcome fits the notion that CRM implementation 
should be a bottom-up initiative, and that top-down endorsement may 
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suffice. The observed association between participation of 90-100% of 
the department's operational managers and strong clinical leadership 
with increased Perceptions of Management scores may be explained in 
part by an increased professional's understanding of the manager's role, 
but may also have helped managers to adopt the disengaged but sup- 
portive stance necessary to facilitate the CRM-initiative. An active core 
group after 1 year was not related to improved SAQ-NL scores, which 
may indicate that transferring the responsibility for implementing the 
“points for improvement” to the regular department’s management team 
might be a viable option in future initiatives. The secondary objectives 
were to investigate what healthcare professionals themselves want to 
improve to increase safety and reduce medical error after receiving CRM 
training (points for improvement), and to explore through observations 
what possible barriers exist to transfer of training to the work floor. 
The most important points for improvement that the participants reported 
were mandatory briefing & debriefing, lowering the threshold for per- 
formance feedback, reducing noise levels, improve the discussion after 
traumatic events, improve and maintain SOP’s, address leadership during 
procedures, and to work towards a non-blaming working environment. 
The process observations yielded several barriers to CRM implementation, 
including perceived time constraints, team breakup, incomplete CRM 
training, CRM core groups that went dormant due to increased work pres- 
sure or lack of initiative, lack of follow-up training of new staff due to lack 
of planning or budget, and insufficient visible commitment from the 
department's management. A possible reason for the transfer of training to 
the work floor to be problematic in some cases could be that CRM training 
is derived from aviation and has not yet been sufficiently adapted to the 
medical context? 


The results from points for improvement and process observations underline 
the importance of mandatory briefing, debriefing and structured 
patient handover skills. Furthermore clinical professionals perceive the 
need for improved leadership/followership and performance feedback 
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skills. Hospital leaders have a key role in creating a working environment 
where learning from mistakes or near misses becomes a daily routine. 
Having hospital leaders participate in CRM-training would be a double- 
edged sword: they would be more aware of Human Factors and, according 
to this study, enhance their managerial position to boot. 


Limitations 

The current observational study has no control group(s) in which repeated 
SAQ-NL measurements were performed without any intervention. Also, 
though it yields positive results, the data does not allow an explanation 
for the exact reasons for these improvements. Furthermore, the SAQ-NL 
response rate post-implementation was relatively low (41%). While low 
response rates may increase the risk of a non-response bias, results from 
“low” response-rate surveys may still accurately represent attitudes of the 
population! 

This appears to be the case in the current study as well, as the third follow- 
up SAQ-NL measurement, which resulted in an adequate response rate, 
showed a continuation of the previous trend. The fact that only two 
hospital departments were represented in the third follow-up measure 
holds the risk of a survey selection bias, which cannot be excluded. 


Future research into the effects of CRM 

More longitudinal studies are needed to see when the effects of CRM 
decline to provide information on when refresher training is appropriate. 
CRM was suggested to increase autonomy in the introduction but this was 
not tested. This is a possible avenue for future research to investigate what 
mechanisms make CRM effective for Job Satisfaction. 


Conclusions 

This study shows that CRM implementation is associated with an improved 
perceived safety climate. Furthermore, our explorative analyses suggest 
that especially integration of simulation training is an important 
determinant of successful CRM implementation. Finally, mandatory 
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(de)briefings as intrinsic part of critical procedures and enhanced 
techniques for performance feedback are considered important points 
for improved care by health care professionals. Human Factors awareness 
in healthcare in the Netherlands is still in an immature stage, with efforts 
aimed at changing the way a clinical department works. Training initiatives 
are facultative, local, and non-standard. For this to improve a multi- 
disciplinary CRM-training curriculum should be agreed upon by the 
medical boards, and made an essential part from undergraduate education 
upward for all (para)medical professionals. 
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Tables 


Table 1 
Demographic, Hospital, Department, and Position Frequencies per 
Measuring Moment. 


EEE 


Variable Category Measure 
m Baseline Post Follow-up 
(n = 1412) (n = 558) (n = 270) 
Gender Male “443 (31.4%)  171(30.6%) 63 (23.3%) 
Female 939 (66.5%) 378(67.7%) 205 (75.9%) 
Age? s20 25 (1.8%) 14 (2.5%) - 
21-30 242 (17.1%) 95 (17.0 %) 32 (11.9%) 
31-40 373 (26.4%) 134 (24.0%) 78 (28.9%) 
41-50 404 (28.6%) 164 (29.4%) 70 (25.9%) 
>50 352 (24.9%) 174 (26.3%) 89 (33.0%) 
Hospital Academic (3/2/1) 451 (31.9%) 244 (43.7%) 220 (81.5%) 
Teaching (7/7/1) 961 (68.1%) 314 (56.3%) 50 (18.5%) 
Department? MCU (1/1/0) 27 (1.9%) 6 (1.1%) - 
ICU (2/2/1) 270 (19.1%) 179 (32.1%) 220 (81.5%) 
OR (5/5/0) 583 (41.3%) 233 (41.8%) - 
CCL (3/2/0) 167 (11.8%) 45 (8.1%) - 
ER (2/2/1) 138 (9.8%) 74 (13.3%) 50 (18.5%) 
CCU-HFA (2/1/0) 77 (5.5%) 21(3.8%) - 
RTX (1/0/0) 11 (0.8%) - - 
DGH (1/0/0) 36 (2.5%) - - 
Pharmacy (1/0/0) 34 (2.4%) - - 
Obstetrics (1/0/0) 
69 (4.9%) 
Position Nurse’ 743 (52.6%) 355 (63.6%) 228 (84.4%) 
Resident 101 (7.2%) 45 (8.1%) 5 (1.9%) 
Att. physician? 256 (18.1%) 106 (19.0%) 22 (8.1%) 


Note. * Age categories in years; ? Between parentheses are numbers per (pre/post/follow-up) 
measure; ° MCU = Medium Care Unit, ICU = Intensive Care Unit, OR = Operating Room, CCL = 
Cardiac Catheterization Lab, ER = Emergency Room, CCU-HFA = Coronory Care Unit — Heart First 
Aid unit, RTX = Radiotherapy, DGH = Department of Gastroenterology and Hepatology; * Nurse 
category consists of nurses, operating room technicians, and anesthesiology technicians; ° Att. 
physician = Attending physician. 
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Table 2 


Means, Standard Deviations, Contrasts, and Effect Size for Safety 
Climate Domains per Measuring Moment 


Domain 


Teamwork 
Climate 


Safety Climate 


Job 
Satisfaction 
Stress 
Recognition 
Perceptions of 
Management 
Working 


Conditions 


Baseline 


M (SD) 


65.0 (14.2) 


61.5 (14.2) 


65.0 (16.4) 


49.2 (17.9) 


48.0 (14.9) 


56.2 (15.9) 


Post 


M (SD) 


67.7 
(13.9) 
63.6 
(13.8) 
64.8 
(16.2) 
52.7 
(17.5) 
53.4 
(16.3) 
56.1 


(16.0) 


Measuring Moment 


p' d’ 
< 1 
001 9 
< 1 
.002 5 
.87 0 

08 

.001 2 
0 

< 3 
„001 6 
„98 0 

01 


Follow-up 

M (SD) p” 
74.53 < 
(13.5) 001 
71.0 < 
(12.4) 001 
74.4 < 
(13.5) .001 
54.9 .036 
(19.3) 
55.9 .082 
(14.1) 
64.9 < 
(14.0) .001 


.57 


64 


12 


17 


58 


Note. Pre n = 1412, Post n = 558, Follow-up n = 270; d = Cohen's d, calculated with pooled standard 
deviation; ' Pre vs. Post; * Post vs. Follow-up. 
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Table 3 


Overview of Most Cited Points for Improvement 


Subject 
Situational 


Awareness 


Communication 


Stress 


Group 
Processes 


Leadership 


Human Error 


Category 
Briefing / 
debriefing 
Handover 
Equipment 
Assertiveness in 
escalation 


Names 


Noise reduction 


Closed-loop 


Address behavior 


Aftercare 


Recognition 


Conflict 


SOP 


Unambiguous 


Safe reporting 


Example 


Discuss the days operations program with 


the entire team before starting the day and 
adress particularities. 

More structured handover of patients. Too 
much information does not get transferred. 
Standardization of monitor colors, e.g. 
saturation always blue. 

Implement 3 levels of escalation in 
potentially unsafe situations; 1. | [think, 
see, etc.], 2. l'm not comfortable, 3. Stop 
this procedure. 

Make sure we know everybody by name 
during a procedure. 

Procedure room needs to be more quiet. 
There is too much talking and there are 
often too many people in a limited space. 
Confirm medication with an open-ended 
question; “what is this?” 

Address wanted and unwanted behavior 
and give proper performance feedback. 
Debriefing of traumatic experiences with 
the other team members. 


Address personal limitations in the team; is 


everybody ‘fit to fly’? 
More talking with each other and less 


about each other; don't let irritation become 


conflict. 

Describe fixed roles, tasks, and 
responsibilities when receiving a trauma 
patient. 

Speak out on who is in charge in critical 
situations. 

Create a compartment in which we can 
learn from others’ mistakes and near- 


misses. 


Frequency 


78 (83%) 


29 (31%) 


11 (12%) 


60 (64%) 


33 (35%) 


32 (34%) 


25 (27%) 


15 (16%) 


22 (23%) 


15 (16%) 


16 (17%) 


12 (13%) 


35 (37%) 


35 (37%) 


Note. N = 94 lists; Only categories that appeared in 10 or more of the lists with points for improvement 
are displayed; SOP = Standard Operating Procedures. 
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Figures 


Teamwork Climate 
~ Safety Climate 





© - Job Satisfaction 
8 Stress Recognition 
~ Perceptions of Management 
- Working Conditions 
Figure 1 


Development of safety climate domains over two years. 
Mean values are depicted. Note that a limited section of the scale is 
displayed to prevent cluttering of lines. 
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Figure 2 

Moderating effects of success factors on safety climate domains. 

Success factors are indicates in the graphs. Mean values are depicted on the 
y-axis. Note that a limited section of the scale is displayed to prevent 
cluttering of lines. POM = Perceptions of Management; WC = Working 


Conditions; SC = Safety Climate. 
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6.2 
Implementation challenges 


Based on: 


Implementation Challenges of Crew Resource Management 
MHTM Haerkens 

CJ Lorraine 

TC Oud 

P Pickkers 

JG van der Hoeven 


J ICU Management & Practice 2016; 16(2):88, 90-91 
Never waste an opportunity to debrief (invited review) 
J Lemson 


MHTM Haerkens 


Critical Care Medicine 2014; 42:7, 1740-1741 
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Summary 


CRM Human Factors awareness training is a useable tool to improve patient 
safety in clinical medicine and may fill a void in medical education curricula. 
However, changing traditional ways of working and improving professional 
interaction remains a challenging task. 

Hands-on experience with CRM in more than 19 high-risk clinical depart- 
ments has yielded insight into the factors impacting CRM implementation 
into the clinical environment. 

While introducing CRM as a clinically-led, non-facultative and safe 
(confidential) multidisciplinary teamwork format may lead to improved 
safety climate and clinical performance in a department, we find that 
these effects will only be sustainable if actively supported by the hospital 
administration. Ultimately, a national CRM-training and evaluation standard 
is considered essential. 
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Introduction 


Performing high risk, time critical procedures, including CPR, is challenging. 
Although mortality may be high, the majority of survivors show a favorable 
neurological outcome1. Importantly, clinical outcome following cardiac 
arrest is not only determined by the technical and procedural proficiency 
of the CPR code team and department logistics? but also by the non- 
technical performance of the professionals involved. This includes team- 
related dynamics, as discussed in chapter 1 of this thesis. Optimal leader- 
ship and followership behavior will strongly influence CPR team results. 
Furthermore, adequate and timely performance feedback during the 
procedure may help avoid mistakes. Also, structured performance feed- 
back after procedures (debriefing) may identify lessons to be learned and 
enhance future team performance and outcome.*>* 

The Human Factor (HF) awareness concept entitled Crew Resource 
Management (CRM) focuses on teamwork, threat and error management, 
and blame-free discussion of human mistakes. This concept originated in 
the aviation domain, but has clear potential for the clinical environment, 
especially in high-risk departments.’”® 

In aviation, CRM is a multidisciplinary non-technical skills standard for 
aircrew. Any CRM-training has to meet Federal Aviation Authority (FAA) or 
European Aviation Safety Agency (EASA) regulations.” Not only do these 
regulations define the various subjects and the extent to which each 
subject should be discussed, but they also set limits for training duration, 
refresher training frequency and trainer requirements. 

Evidence is increasing that CRM is also a promising tool for culture change 
in high-risk clinical departments. Unfortunately, currently, standardization 
in medical HF-training initiatives is lacking. As a consequence, large 
variations in curriculum, duration, intensity, feed-back and follow-up 
support exist. 
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Our approach 


To reach a credible national standard for CRM in medical training and 
operations an evidence-based curriculum is key.'' Our research includes 
data from CRM “culture-interventions” in more than 19 high-risk clinical 
departments in university and training hospitals, and investigates the 
impact on safety climate and clinical performance. 

Clearly, implementing CRM in departments whose professionals have had 
no previous formal HF education requires more than just training. There- 
fore, we introduced a three-phase “intervention” approach, consisting of 
a preparation, training, and implementation phase, including feedback of 
results. 


The three-month preparation phase kicks off with a plenary session to 
inform all training participants on the upcoming CRM-training and 
implementation flow. Even though CRM relies on intrinsic motivation to be 
effective, we encourage the department leadership to clarify to all staff in 
advance that CRM will become the non-facultative professional standard, 
serving as a yardstick for professional evaluation. 

Subsequently, the department's safety climate is assessed using the SAQ- 
NL (as described in chapter 2), onsite process observations are conducted 
by CRM-trainers and video-footage of critical communication moments is 
gathered. 


The training phase’s multidisciplinary sessions are performed within a 
three-month window to maximize impact and held at a facility at some 
distance from the hospital to minimize interference. The maximum amount 
of participants for training sessions is 15. Sessions are conducted by two 
trainers with a credible operational and scientific background. 

The well-developed two-day standard course includes lectures on HF and 
principles of CRM, multiple interactive sessions using realistic data (such as 
case studies), and video footage from the trained department. 
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The training emphasizes nine key areas: situational awareness and 
recognition of adverse situations; human errors and non-punitive response; 
communication and briefing and debriefing techniques; providing and 
receiving performance feedback, management of stress; workload and 
fatigue; creating and maintaining team structure and climate; leadership; 
risk management; and decision-making."' 

Each training group is challenged to produce a shortlist of practical 
“action points” to be converted into practical clinical tools in the following 
implementation phase. 


The one-year implementation phase aims to integrate and develop the 
CRM way of professional interaction within the department. The intended 
culture change is supported by additional implementation measures like 
the formation of a CRM “core group” and scenario-based team training 
sessions. 

The core group, consisting of volunteer clinical professionals, is tasked with 
the coordination of the efforts to convert the CRM action points into clinical 
practice.11 They receive additional coaching during the implementation 
year. Clearly, it is important for the clinical leadership to empower the core 
group activities, thereby creating joint “ownership”. 

Scenario-based team training, also called simulation, is encouraged as a 
follow-up measure to reinforce the effect of CRM-based culture change. 
Scenario-based team training creates a zero-risk environment that 
allows medical teams to practice high-risk, low-frequency events without 
endangering patients. Simulation can be done in an artificial “laboratory” 
environment or as “in situ” training, which is conducted on actual 
patient care units, involving actual health care team members and actual 
organization processes."' If well debriefed, simulation has many advan- 
tages, but if used as a stand-alone measure without the basis of CRM- 
training there is a risk of focusing too much on individual technical skills 
and single-task performance. This could result in a limited impact on overall 
safety. 
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The basis for scenario-based team training in healthcare is the identifi- 
cation of the domain-specific team skills required to manage routine and 
emergency scenarios. Based on aviation simulation experience we suggest 
using two separate phases of simulation training: the first level of training 
mainly focuses on technical and procedural skills. Only after having 
completed CRM-training, participants may be scheduled for second level 
simulation training, which emphasizes nontechnical (team)performance. 


Defining and measuring success 

As the ultimate goal of Crew Resource Management is to improve patient 

safety, defining elements of CRM success could be: acceptance by the 

professionals and management, improved safety climate, and ultimately to 

a decrease in complications and improved patient outcome. 

However, objective determination of these factors has proved challenging. 

We encountered the following limitations: 

e Training evaluation scores could be considered as indicators of accep- 
tance by participating professionals and managers, but only indirectly 
so. 

+ Safety climate was measured in all projects with the Safety Attitudes 
Questionnaire (SAQ-NL, see chapter two), but response rates varied 
extensively. 

e Reliable determinations of complications proved challenging in 17 out 
of 19 clinical departments, as registration is performed by specialisms 
independently of each other, and data are rarely traceable to a specific 
department (e.g. the Emergency Department or Operating Room). The 
only departments that supplied us with solid complication data were 
the ICU's, participating in the Dutch National Intensive Care Evaluation 
(NICE) that records a predefined set of complications." 


Experience with CRM-implementation 

Our experience with CRM-implementation in 19 high-risk clinical depart- 
ments in the Netherlands is encouraging: First, the training evaluation 
scores were high in all trained departments. We consider this an indication 
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that the aviation-derived training concept was highly appreciated by 
medical professionals and identified as an important part of professional 
self-regulation. Second, as discussed in chapter 6.1, CRM was associated 
with a positive effect on all domains of perceived safety climate. Third, as 
discussed in Chapter 4 of this thesis, we found CRM-implementation in the 
ICU was associated with a reduction in serious complications and lower 
mortality in critically ill patients (next to an improved safety climate)? Due 
to the aforementioned lack of registration of complication in other depart- 
ments, reliable reproduction of these data in other settings was not feasible. 
Still, also important barriers to implementation"? were encountered on a 
financial, organizational, managerial and professional level. Ever increasing 
financial and time constraints challenge the funding and time investment 
crucial for CRM training and implementation efforts. Although one of our 
projects was awarded the 2013 Dutch Health Inspectorate’s national Patient 
Safety Award, to date Dutch national medical societies and health insurers 
have not yet agreed on funding for a national Human Factors training 
curriculum for healthcare professionals. 

In our experience, traditional hospital organizational structures and exis- 
ting department cultures may induce a general reluctance to change ways 
of working, especially if they are thought to involve additional effort. The 
required leadership by example from senior staff members unfortunately 
proved to be more exception than rule. Furthermore, integrating 
managers/administrators into the CRM-implementation process proved 
challenging as expectations toward management's role varied. CRM 
trainees expected management to facilitate CRM implementation by 
allowing sufficient time and adequate funding, but frequently under- 
estimated the requirement for active support from and cooperation with 
managers, both local and corporate. 

While clinicians were in general enthusiastic about initiating and leading 
CRM activities, they often became more ambivalent if it led them into more 
‘managerial’ activity. This deep-seated concern about managerial work 
remains an obstacle to the development of HF in healthcare. Our approach 
to introduce CRM as an initiative led by clinicians requiring a disengaged 
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but supportive stance of the hospitals’ management probably improved 
the acceptation by the professionals, but this bottom-up approach may, 
in some cases, have caused managers to adopt a negative — “not invented 
here” — attitude and disengage from the implementation process, and thus 
weakening the initiative. 


Hands-on experience from the aforementioned 19 CRM implementation 
projects, as well as the results from the longitudinal study described in 
chapter 6.1, helped us identify some key success factors as well as threats to 
success that may be of use to future clinical CRM initiatives. Table 1 contains 
an overview of empiric success factors and threats to success. 


Conclusions 

Aviation-based CRM Human Factors awareness training is a useful tool 
in clinical medicine that increases perceived safety climate and has the 
potential to improve clinical outcome. However, adapting the organiza- 
tional context to fully integrate the Human Factors principles into daily 
operations remains challenging. Nevertheless, with the identification of 
success factors as well as threats to success, successful implementation is 
possible. Currently, an (inter)national standard for medical CRM-training 
and evaluation is lacking. For CRM to truly impact safety for all patients, 
it should be an obligatory component of healthcare. Therefore, a national 
standard should be agreed upon, from under-graduate curricula to daily 
clinical operations. This standard will facilitate compatibility, safety and 
quality in multidisciplinary care. 

Furthermore, it will make it easier to compare educational service providers. 
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Tables 


Table 1: Success factors and threats to success 


Success factors background 





The initiative is clinically-led Helps convince other clinicians of the relevance 
of HF awareness in daily practice. 

Training is delivered largely by clinicians | Facilitates acceptance of the curriculum and 
case discussions. 





Confidentiality is maintained throughout | Essential for discussions on sensitive subjects, 

the intervention such as case discussions. 

Participation is non-facultative for all staff | Helps all professionals involved in the care 

members, students and operational process to recognize and appreciate changes 

managers resulting from the CRM training. 

Training is conducted in multi- Increases understanding of each other's 

disciplinary groups problems and limitations whilst making it 
impossible to apportion blame to groups not 
represented in the training. 





Hospital administrator is involved Helps position CRM as multidisciplinary clinical 
standard, and “make things happen” 





Hospital managers take a supportive, but | Facilitates organizational change, and helps 
“hands-off” stance clinical professionals appreciate the importance 
of managerial support. 


Threats to success background 


Poorly managed expectations of the May result in the idea that CRM training in and 


project of itself will yield a culture change, and lead to 


underestimation of the required implementation 
efforts. 





Lack of a solid implementation plan and | Improvement is unlikely to be seen within one 
realistic timeline year and requires dedication and tenacity from 


all involved 11° 


Delay in initiating timely and noticeable Makes it harder for professionals to relate the 
Insufficient accountable operational If clinical leaders do not set the example, safety 
Disengagement of hospital administrator | Will frustrate bottom-up initiatives from the work 


CRM is used as “window dressing” May result in inadequate follow-up and very 


limited commitment of caregivers. 
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7.1 
Summary 
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Summary 


Despite modern equipment, increasing emphasis on patient safety, and 
excellent training facilities, medical care frequently results in unintentional 
harm to patients. Reliable studies on medical error are scarce, as they need 
to rely on data extractable from documented inpatient health records, 
which makes identifying the key factors in safe care a challenging task. 
Human Factors (HF) appear to play an important role in adverse events, 
especially in departments where high risk, time critical procedures on 
vulnerable patients are performed in a multidisciplinary team setting. 

As perceived safety climate is positively related to safety outcomes both 
in hospital settings and other high-hazard fields, a sound safety climate is 
considered essential. 


This thesis has focused on HF and critical team performance in clinical 
medicine. We validated an open source safety climate assessment tool (the 
SAQ-NL) and developed an aviation-derived team concept for the clinical 
setting (Crew Resource Management - CRM), including a training sylla- 
bus and implementation plan. In addition, we performed the CRM safety 
climate intervention in multiple hospital departments. and assessed its 
impact on patient safety. 


Chapter one provided an introduction into the educational and opera- 
tional safety issues of the clinical healthcare system in the Netherlands as 
well as an outline of the thesis. 


In chapter two safety climate in a variety of clinical departments in Dutch 
hospitals was investigated and yielded a validated open-source instrument 
to measure safety climate in the Dutch clinical setting, the Safety Attitudes 
Questionnaire-NL (SAQ-NL). As removal of one item resulted in an 
increased reliability of the Working Conditions dimension, revision or 
deletion of this item should be considered. The results from this study 
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provide researchers and practitioners with insight into safety climate in a 
variety of departments and functional positions in Dutch hospitals. 


In chapter three the existing literature on CRM and its application in critical 
care medicine was reviewed. Furthermore, both the framework of our HF 
awareness training curriculum specifically aimed at care teams, as well 
as a three-phase structure for the CRM safety climate intervention was 
described. Even though evidence of the impact of CRM on medical errors 
and patient outcome is still scarce, the parallels between the critical 
processes in aviation and Intensive Care (ICU) suggest that a well-adapted 
medical CRM training has potential for the ICU environment. 


Chapter four described a study into the impact of a CRM-intervention in 
a large ICU on patient outcome. The CRM-implementation was associated 
with a reduction in serious complications and lower mortality in critically ill 
patients. These clinically relevant effects paralleled a positive impact on the 
perceived safety climate by the health care providers. 


In chapter five, the effects of a CRM-intervention in the Trauma Room 
setting of a level 1 Emergency Department was described. Although CRM- 
implementation in the ED was associated with an improved safety climate, 
the time spent by trauma patients in the ED increased. 


In chapter six, the impact of CRM implementation on safety climate in 17 
high-risk clinical departments was investigated, and this impact was related 
to perceived implementation success factors as well as threats to success. 
It was found that CRM implementation is associated with an improved 
safety climate. Also a shortlist of factors associated with successful CRM 
implementation, as well as factors associated with failure of implemen- 
tation, was provided. Furthermore, points for improvement that 
healthcare professionals wanted to implement after receiving CRM 
training, and possible existing barriers to transfer of the CRM training to 
the work floor, were identified. 
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Chapter seven summarized all research findings and discussed current 
issues and future perspectives for CRM. Although aviation-based CRM/ 
Human Factors awareness training may prove a useful tool in medicine, 
adapting the medical organizational context to fully integrate the Human 
Factors principles into daily health operations remains a challenge. 
Operational, educational and inspectorate/judicial aspects of this 
challenge are discussed. 

It is concluded that only after an national medical CRM-training and 
operational standard is agreed upon, the CRM Human Factors awareness 
program may mature into an effective component of under- and post- 
graduate (para)medical curricula. 
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7.2 
Future perspectives 


Based on: 


Team aspects of Patient Safety 

MHTM Haerkens 

Dutch Patient Safety Manual 2016; Part 5, Chapter 23. 
ISBN: 9789058983060 


Crew Resource Management 

ECTH Tan 

MHTM Haerkens 

Acute Healthcare 2014; Chapter 1.4. ISBN: 9789035236233 
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Future perspectives 


Despite modern equipment, permanent education, continuing research 
and excellent training facilities medical care frequently results in uninten- 
tional harm to patients. Medical error is currently estimated to be the third 
leading cause of death in the United States, with over 250,000 patients 
having died as a result of preventable adverse events in 2013.' The actual 
numbers may even be higher, for underreporting is likely as standardized 
root cause analysis, reporting of inpatient deaths and national data sharing 
is limited.’ Although some medical errors can be attributed to technical 
malfunctions and failing equipment, many find their roots in the area of 
Human Factors (HF). It may therefore be reasonable for the healthcare 
community to improve HF awareness and non-technical skills in order to 
reduce medical error and increase safety. 


Safety Climate 

Each healthcare professional carries the responsibility to operate 
competently, safely and in the best interest of the patient. However, as safe 
care relies on the interdisciplinary performance of a care team as much 
as on individual expertise, a “just” safety culture may prove a sound basis 
for better patient outcomes.” As described in chapters two and three of 
this thesis, safety climate is positively related to safety outcomes (both in 
hospital settings and other high-hazard fields), responsive to interventions, 
and can be reliably determined through surveys. Therefore it may be 
valuable for all high-risk clinical departments to gauge their safety climate 
ona regular basis. 


Future of CRM 

There are critical needs to be met for CRM to have a role in national patient 
safety: additional convincing research, a clear mandate and sufficient 
resources.” 
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The first most obvious need is the access to more convincing high-quality 
research. Despite encouraging results from local healthcare CRM- 
interventions, there is a lack of studies establishing a credible, direct 
cause-and-effect relationship between CRM training and safety. These 
studies should include all levels of Kirkpatrick's evaluation framework 
with an accent on observed explicit professional behavior and patient 
outcomes, data that may not be readily available for multiple reasons (see 
below). 


A clear mandate - on multiple levels - is a second necessity, as professional 
safety attitudes are forged during training and influenced by daily practice 
and even legal context. Therefore, it would make sense to allow a national 
organization of healthcare professionals, such as the Dutch Federation of 
Medical Specialists (Federatie Medisch Specialisten - FMS), the mandate to 
focus future efforts to improve safety climate on operational patient care, 
educational system as well as inspectorate/judicial level. 

1. On an operational level several non-technical issues have an impact on 
care team performance. First, leadership/followership roles in health care 
teams should be re-defined, focusing on team role and situational compe- 
tence, not traditional hierarchic position. Traditional hierarchy negatively 
influences cooperation within care teams (eg. doctors vs. nurses, junior vs. 
senior staff), between different clinical specialties, and is also noticeable 
in the relationship between hospital administrators and medical profes- 
sionals. For this to improve, leadership and followership skills should be 
considered core competences for medical professionals. CRM may serve as 
a common basis for this skill set. Second, as discussed in chapter 6, struc- 
tured debriefing should become a predictable, non-facultative part of 
every critical operational process. Debriefing is a powerful tool to identify 
lessons to be learned shortly after a procedure, with the sole purpose to 
improve future individual and team performance and prevent an incident 
or near-miss from ever occurring again. Third, failure to share errors and 
adverse events with colleagues remains an important problem in care, as 
other team members can only learn from errors that have been disclosed.* 
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Differences between doctors and nurses in the willingness to report 
unintended events exist.” Additionally, patients and caregivers may have 
different ideas about what constitutes a medical error or adverse event* 
or which information needs to be shared with whom. To achieve the 
aforementioned “just culture” national healthcare organizations should 
agree upon a national multidisciplinary standard in medical team 
dynamics. This may require changes to the educational system and the 
inspectorate/judicial context. 

2. The medial educational system is traditionally more focused on 
individual craftsmanship and professional autonomy than on non-technical 
skills such as leader- (and follower-) ship, team dynamics and communi- 
cation. To date, many university and training hospitals have no comprehen- 
sive HF awareness curriculum and lack sufficient HF expertise. This often 
results in outsourcing of training. Following encouraging reports on the 
effects of HF-awareness training in clinical medicine many commercial 
CRM-derived training initiatives arose - without any standard for trainers, 
training content and duration, or implementation approach. 

For this to change, Dutch healthcare organizations and authorities should 
first aim to achieve consensus on a robust standard for HF awareness 
teaching and skill evaluation for (para)medical professionals. Second, 
recognizing the traditional barriers that exist between doctor's and nurse's 
training institutes, it should strive towards a truly multidisciplinary under- 
and postgraduate HF awareness training curriculum. As mentioned earlier, 
a robust national organization of healthcare professionals (like the FMS) 
should be mandated to coordinate these efforts. 

3. Finally, on an inspectorate/judicial level there are challenges that need 
to be addressed: in contrast to the aviation domain, the Dutch medical 
incident reporting system has currently no well-defined “blame-free” legal 
compartment with the sole purpose to enable all professionals to learn 
from errors that have occurred. As a consequence, every safety report may 
end up directing undesired attention from healthcare and judicial authori- 
ties towards the medical professional. This does not facilitate low-threshold 
safety reporting, despite current efforts to establish incident reporting and 
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learning systems in healthcare.” As previously discussed, authorities should 
consider initiating a non-blaming incident report database, with oversight 
separated from both the Health Inspectorate and legal system. Beyond 
the scope of this thesis, but a potential factor nonetheless, is the recent 
development that Dutch personal injury lawyers are allowed to adopt the 
no cure no pay — fee system for law suits during a 5-year experimental 
period 

ending January 1, 2019.° If this revenue model were to become permanent, 
the perception that it may encourage an increased motivation for law 
suits (and the concurrent fear of “ambulance chasers”) would not help in 
lowering the disclosure threshold for healthcare professionals. 


An effective national CRM initiative also requires sufficient financial and 
human resources, as CRM research, development and implementation is 
very labor intensive. 

First, resources are needed to standardize training evaluation and to 
properly train sufficient trainers and evaluators. Additionally, if CRM efforts 
were to continue as local hospital initiatives with early adapters, outsour- 
cing training to commercial organizations would remain too costly. Ade- 
quate, centralized funding would facilitate a coordinated development of 
training content, integration into existing undergraduate curricula as well 
as continuous medical education.’ Finally, establishing the cost effective- 
ness of CRM in healthcare is a very challenging endeavour, as both CRM 
training and implementation strategies and calculation methods differ. A 
representative overview of medical errors as a cause for patient harm (and 
their costs) is usually not available. 

The few studies aiming to establish the cost effectiveness of CRM in health- 
care show variable results. Moffatt-Bruce et al conclude that CRM presents 
a financially viable way to systematically organize quality improvement,’ 
while Kemper et al found that their CRM implementation was not cost 
effective because they were unable to show a beneficial effect on patient 
outcome.” 
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Meanwhile, the devastating loss of human life and colossal cost (in the 
US alone estimated at $1 trillion each year") resulting from preventable 
medical error may warrant continuation of HF awareness investments. 


Future research 

This thesis may serve as a basis for future research to further enhance the 
SAQ-NL safety climate questionnaire or CRM training content. Furthermore, 
additional longitudinal studies are needed to ascertain the impact of CRM 
on patient outcome in different clinical settings, and to identify when the 
effects of CRM decline to provide information on when refresher training 
is appropriate. To evaluate cost effectiveness we would encourage future 
studies with larger study populations and longer follow-up. 

The Crew Resource Management Human Factors awareness program may 
mature into an effective component of under- and postgraduate (para)- 
medical curricula. But only after an (inter)national medical CRM-training 
and operational standard is agreed upon. 


It is time that we should appreciate the human factor. 
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8.1 
Summary in Dutch - Samenvatting 
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Samenvatting 


Ondanks moderne apparatuur, evidence based standaarden, veiligheids- 
instrumenten, kwaliteitsindicatoren, veiligheidsmanagementsystemen, 
certificeringen en audits resulteert medische zorg nog te vaak in 
onbedoelde schade voor de patiënt. 

Betrouwbare studies over medische fouten zijn schaars, en maken gebruik 
van data afkomstig uit medische statussen. Dit maakt het identificeren 
van leermomenten een uitdagende taak. Human Factors (HF) spelen een 
belangrijke rol in incidenten in alle hoog-risico processen. Binnen de 
zorg gaat het dan speciaal om afdelingen waar risicovolle, tijdskritische 
procedures worden verricht door multidisciplinaire teams op kwetsbare 
patiënten. 

Een optimaal veiligheidsklimaat is belangrijk, aangezien het positief 
correleert met procesveiligheid, zowel in de medische zorg als andere 
hoogkritische sectoren. 


Dit proefschrift is gericht op het verband tussen HF en het functioneren van 
hoogkritische teams binnen de klinische patiëntenzorg. 

Een instrument ter meting van het veiligheidsklimaat werd gevalideerd 
(SAQ-NL), en een uit de luchtvaart afkomstig teamtrainingsconcept werd 
toegesneden op de ziekenhuisomgeving (Crew Resource Management 
- CRM), inclusief een training syllabus en implementatieplan. De CRM- 
cultuurinterventie werd vervolgens toegepast in meerdere ziekenhuis- 
afdelingen, en de invloed op patiëntveiligheid werd onderzocht. 


Hoofdstuk 1 geeft een introductie in de uitdagingen rondom patiënt- 
veiligheid op opleidings- en operationeel gebied, en biedt tevens een 
overzicht van het proefschrift. 


In hoofdstuk 2 werd het veiligheidsklimaat onderzocht op verschil- 


lende ziekenhuisafdelingen met een aan de Nederlandse taal en context 
aan-gepaste wereldwijd gebenchmarkte vragenlijst (Safety Attitudes 
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Questionnaire - SAQ) teneinde tot een gevalideerd, en voor eenieder 
beschikbaar, meetinstrument te komen voor veiligheidsklimaat in de 
Nederlandse klinische setting (SAQ-NL). De SAQ-NL liet prima eigen- 
schappen zien. Wel is aanpassing of verwijdering van 1 vraag te 
overwegen, aangezien dat de betrouwbaarheid van de Arbeidsomstandig- 
heden-dimensie zou vergroten. De resultaten van deze studie bieden zowel 
onderzoekers als medische professionals inzicht in het veiligheidsklimaat 
voor een variéteit aan afdelingen en beroepsgroepen binnen ziekenhuizen. 


In hoofdstuk drie werd de beschikbare literatuur over HF en CRM en de 
toepassing binnen hoogkritische medische zorgprocessen besproken. 
Bovendien werd ons CRM opleidingscurriculum voor niet-technische 
vaardigheden en teamfunctioneren in de zorg beschreven, evenals onze 
drie-fasische aanpak voor de CRM veiligheidsklimaat-interventie. Alhoe- 
wel het beschikbare wetenschappelijke bewijs voor het effect van CRM op 
patiéntenzorg nog zeer beperkt is, duiden de parallellen tussen de hoog- 
kritische processen binnen zorg en luchtvaart op het mogelijke nut van een 
goed op de (intensieve) zorg toegesneden medische CRM variant. 


Hoofdstuk vier beschreef een studie naar de invloed van een CRM-inter- 
ventie op de patiëntenzorg. De CRM-implementatie was geassocieerd met 
een afname van ernstige complicaties en verminderde sterfte bij ernstig 
zieke patiënten. Deze klinisch relevante effecten gingen gepaard met een 
verbetering van het veiligheidsklimaat op de afdeling. 


In hoofdstuk vijf, werden de effecten van een CRM-interventie op de 
traumakamer van een level 1 Spoedeisende Hulp (SEH) onderzocht. 
Alhoewel de introductie van CRM op de SEH geassocieerd was met een 
verbeterd veiligheidsklimaat, nam de verblijfsduur van traumapatiënten op 
de SEH toe. 


In hoofdstuk zes werd de invloed van CRM op het veiligheidsklimaat van 
17 hoogrisico klinische afdelingen onderzocht, waarbij deze invloed werd 
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gecorreleerd met empirische succes- en faalfactoren voor implementa- 
tie. De resultaten lieten zien dat CRM implementatie geassocieerd is met 
een verbeterd veiligheidsklimaat, en resulteerden tevens in een korte lijst 
van succes- en faalfactoren voor CRM implementatie. Bovendien werden 
verbeterpunten geformuleerd door zorgprofessionals tijdens alle CRM- 
trainingen (actiepunten) geinventariseerd, evenals barriéres voor het door- 
voeren van geleerde methodieken (en actiepunten) op de werkvloer. 


Hoofdstuk zeven bevatte een samenvatting van alle onderzoeks- 
resultaten en besprak huidige situatie en toekomstige perspectieven 
voor het CRM initiatief. Alhoewel een goed op de zorg toegesneden CRM 
concept veelbelovend is, blijkt het zodanig aanpassen van een zieken- 
huisorganisatie dat de HF-principes volledig worden geïntegreerd in de 
dagelijkse patiëntenzorg momenteel nog uitdagend. Zowel de operatio- 
nele, opleidings, inspectie, als juridische aspecten van deze uitdaging 
werden besproken. 


Er werd geconcludeerd dat Crew Resource Management kan rijpen tot 
een waardevolle component van multidisciplinaire medische zorg en alle 
(para)medische opleidingen, op voorwaarde dat we in staat zijn om tot een 
nationale standaard te komen. Dit proefschrift beoogt daarvoor een basis 
te bieden. 
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STELLINGEN 


behorende bij het proefschrift 
Human Factors and Team Performance 
Het veiligheidsklimaat van een afdeling is te meten. (dit proefschrift) 


Artsen schatten het veiligheidsklimaat op een afdeling hoger in dan 
verpleegkundigen. (dit proefschrift) 


Slechts 1 op de 7 medische professionals voelt zich gesteund door het 
ziekenhuismanagement. (dit proefschrift) 


Crew Resource Management (CRM) is geassocieerd met een verbeterd 
veiligheidsklimaat op zowel de Intensive Care (IC) als Spoedeisende 


Hulp. (dit proefschrift) 


Integratie van scenario training in een CRM-cultuurinterventie is 
geassocieerd met een verbeterd veiligheidsklimaat. (dit proefschrift) 


CRM is geassocieerd met een afname van ernstige complicaties en een 
reductie van mortaliteit op de IC. (dit proefschrift) 


De zorg is een uniek domein, waar klimaatverandering juist wenselijk is. 
Het feit dat op iedere promotie-uitnodiging achter de aanvangstijd 
“precies” staat, geeft aan dat academische punctualiteit niet vanzelf- 


sprekend is. 


De missie van militaire vliegers, traumachirurgen en promovendi: het 
oncontroleerbare controleren. 
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10. You want a surgical team that faces each error, each mishap, straight up, 


names it, and takes steps to prevent its recurrence. 
Francis D Moore 


. Don't ask what the world needs. Ask what makes you come alive, and 


go do it. Because what the world needs is people who have come alive. 
Howard Truman 


. He will win whose army is animated by the spirit throughout all its 


ranks. Sun Tzu, The Art of War 
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PROPOSITIONS 


accompanying the thesis 
Human Factors and Team Performance 
A department's safety climate is measurable. (this thesis) 


Doctors perceive their department's safety climate better than nurses. 
(this thesis) 


Only 1 out of 7 clinical professionals feels supported by their hospital 
management. (this thesis) 


CRM is associated with an improved safety climate both on the 
Intensive Care Unit (ICU) and the Emergency Department. (this thesis) 


Integration of simulation training into Crew Resource Management 
(CRM) implementation is associated with improvements in safety 


climate. (this thesis) 


CRM is associated with a reduction in serious complications and a 
reduction in mortality at the ICU. (this thesis) 


Healthcare is a unique domain where climate change is a good thing. 
The fact that every invitation for a PhD thesis defense states the word 
“exactly” after the starting time indicates that academic punctuality is 


not self-evident. 


The mission of military aviators, trauma surgeons and PhD candidates is 
to control the uncontrollable. 
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10. You want a surgical team that faces each error, each mishap, straight up, 


names it, and takes steps to prevent its recurrence. 
Francis D Moore 


. Don't ask what the world needs. Ask what makes you come alive, and 


go do it. Because what the world needs is people who have come alive. 
Howard Truman 


. He will win whose army is animated by the spirit throughout all its 


ranks. Sun Tzu, The Art of War 
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